-- Logs begin at Tue 2022-10-04 08:04:08 UTC, end at Tue 2022-10-04 09:12:45 UTC. -- Oct 04 08:31:16.819163 dv-cinder-860212-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 04 08:31:16.826472 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 04 08:31:16.828462 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 4 08:31:16 2022] *** Oct 04 08:31:16.828462 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 04 08:31:16.828462 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 04 08:31:16.828462 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: nodename: dv-cinder-860212-1-iscsi Oct 04 08:31:16.828462 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: machine: x86_64 Oct 04 08:31:16.828462 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: clock source: unix Oct 04 08:31:16.828861 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: pcre jit disabled Oct 04 08:31:16.828861 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: detected number of CPU cores: 4 Oct 04 08:31:16.828861 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: current working directory: / Oct 04 08:31:16.828861 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: detected binary path: /usr/bin/uwsgi-core Oct 04 08:31:16.828861 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: your processes number limit is 47680 Oct 04 08:31:16.828861 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: your memory page size is 4096 bytes Oct 04 08:31:16.828861 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: detected max file descriptor number: 2048 Oct 04 08:31:16.828861 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: lock engine: pthread robust mutexes Oct 04 08:31:16.828861 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: thunder lock: enabled Oct 04 08:31:16.828861 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 04 08:31:16.829440 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 04 08:31:16.869646 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: Python main interpreter initialized at 0x563078c359c0 Oct 04 08:31:16.869646 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: python threads support enabled Oct 04 08:31:16.869646 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: your server socket listen backlog is limited to 100 connections Oct 04 08:31:16.869646 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: your mercy for graceful operations on workers is 90 seconds Oct 04 08:31:16.870026 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: mapped 403077 bytes (393 KB) for 2 cores Oct 04 08:31:16.870127 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: *** Operational MODE: preforking *** Oct 04 08:31:16.870397 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: *** uWSGI is running in multiple interpreter mode *** Oct 04 08:31:16.870397 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: spawned uWSGI master process (pid: 87935) Oct 04 08:31:16.870592 dv-cinder-860212-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 04 08:31:16.872500 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: spawned uWSGI worker 1 (pid: 87945, cores: 1) Oct 04 08:31:16.872500 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: spawned uWSGI worker 2 (pid: 87946, cores: 1) Oct 04 08:31:16.872500 dv-cinder-860212-1-iscsi devstack@keystone.service[87935]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 04 08:31:21.139169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:31:21.139169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: from cryptography.utils import int_from_bytes Oct 04 08:31:21.139982 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:31:21.139982 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: from cryptography.utils import int_from_bytes Oct 04 08:31:21.143239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:31:21.143239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: from cryptography.utils import int_from_bytes Oct 04 08:31:21.143681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 08:31:21.143681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: from cryptography.utils import int_from_bytes Oct 04 08:31:21.189857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 08:31:21.190052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 08:31:21.190247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] command line args: [] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 08:31:21.190520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 08:31:21.190699 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ================================================================================ {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 08:31:21.190955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] admin_token = **** {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.191076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 08:31:21.191200 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] config_dir = [] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.191285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 08:31:21.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.191467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] command line args: [] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 08:31:21.191574 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] config_source = [] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.191715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 08:31:21.191799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] debug = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.191894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ================================================================================ {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 08:31:21.191971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.192233 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] admin_token = **** {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.192340 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.192436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] config_dir = [] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.192528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.192620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.192722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.192821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] config_source = [] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.192922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] debug = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.193022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.193123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.193256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] list_limit = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.193351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.193451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] log_config_append = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.193545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.193643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.193766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] log_dir = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.193871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.193979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] log_file = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.194083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.194183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.194284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] list_limit = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.194397 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.194478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] log_config_append = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.194598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.194684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.194762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] log_dir = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.194833 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.194949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] log_file = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.195021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.195142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.195222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.195327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.195447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.195574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.195651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.195818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.195897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.196016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.196097 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.196174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.196294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.196424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.196530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.196614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.196780 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.196864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.197027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.197113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.197270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.197354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.197501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.197766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.197859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.197972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] publish_errors = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.198049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.198207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.198286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.198440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.198620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.198743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.198868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.198959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] publish_errors = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.199103 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.199187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.199337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] standard_threads = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.199432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.199511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.199581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.199758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.199842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.199963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.200071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.200181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] use_journal = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.200290 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] standard_threads = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.200370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] use_json = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.200441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.200510 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] use_stderr = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.200630 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] use_syslog = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.200703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.200824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.200908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.201002 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] use_journal = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.201080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.201161 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] use_json = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.201316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.201441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] use_stderr = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.201515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] use_syslog = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.201590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.201736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 08:31:21.201829 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.201994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.202085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.202233 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.202311 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.202492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.202578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.202730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.202807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.202981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.203057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.203224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.203334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.203465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.203548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.203699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.203776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.203947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.204031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.204187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.204264 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.204429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.204506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.204663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.204753 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.204904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.204980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.205137 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.205219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.205383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.205461 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.205630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.205738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.205892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.205968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.206145 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.206225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.206396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.206474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.206638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.206716 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.206872 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.206950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.207108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.207192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.207343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.207426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.207577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.207656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.207814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.207890 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.208064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.208140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.208293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.208375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.208523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.208600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.208754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.208830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.208999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.209082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.209246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.209374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.209506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.209628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.209812 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.209943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.210067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.210209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.210341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.210484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.210614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.211234 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.211234 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.211466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.211589 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.211712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.211843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.211967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.212091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.212220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.212344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.212479 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.212479 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.212786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.212908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.213031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.213153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.213275 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.213407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.213531 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.213653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.213800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.213924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.214080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.214156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.214227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.214327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.214420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.214519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.214621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.214717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.214815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.214909 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.215002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.215095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.215195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.215295 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.215409 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.215491 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.215642 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.215763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.215848 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.215963 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.216037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.216149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.216232 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.216305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.216458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.216548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.216623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.216695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.216760 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.216828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.216904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.216975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.217055 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.217201 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.217283 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.217437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.217522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.217669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.217807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.217930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.218008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.218166 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.218246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.218436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.218573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.218695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.218826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.218945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.219070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.219202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.219332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.219453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.219578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.219706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.219826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.219958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.220092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.220214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.220363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.220478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.220606 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.220738 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.220871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.220993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.221118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.221239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.221364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.221494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.221630 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.221795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.221923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.222048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.222172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.222305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.222437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.222565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.222689 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.222814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.222939 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.223063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.223194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.223319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.223444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.223577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.223702 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.223832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.224047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.224169 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.224294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.224419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.224549 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.224682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.224817 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.224952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.225084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.225205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.225280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.225351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.225421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.225499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.225572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.225650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.225762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.225835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.225906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.226028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.226115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.226206 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.226278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.227027 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.227117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.227199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.227272 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.227396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.227769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] auth.external = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.227844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.227942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.228023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.228097 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.228176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.228249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.228325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] auth.external = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.228397 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] auth.password = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.228519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.228596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] auth.token = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.228675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.228754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.228832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.228932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.229037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] auth.password = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.229183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.229270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] auth.token = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.229417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.229497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.229667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.229815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.229939 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.230031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.230174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.230252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.230412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] credential.caching = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.230560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.230685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.230818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.230945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.231071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.231199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] credential.caching = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.231326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.231452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.231589 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.231714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.231852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.231987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.232112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.232238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.232376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.232503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.232629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.232754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.232879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.233027 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.233154 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.233285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.233412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.233548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.233674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.233842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.233970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.234096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.234222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.234354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.234480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.234613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.234740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] federation.caching = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.234866 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.234991 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.235117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.235240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.235384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] federation.caching = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.235512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.235639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.235766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.235892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.236016 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.236150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.236281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.236407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.236533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.236659 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.236786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.236920 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.237056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.237189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.237316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.237446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.237579 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.237728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.237854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.caching = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.237980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.238106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.238233 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.238358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.238557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.caching = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.238683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.238811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.238948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.239075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.239207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.239341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.239464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.239590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.239724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.239850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.239977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.240111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.240237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.240318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.240442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.240572 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.240696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.240831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.240954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.241076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.241198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.241320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.241441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.241569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.241710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.241842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.241987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.242118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.242242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.242384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.242513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.242636 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.242759 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.242880 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.243009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.243132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.243254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.243375 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.243498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.243619 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.243749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.243878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.244019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.244152 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.244276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.244276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.244513 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.244643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.244767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.244889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.245011 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.245133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.245263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.245394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.245518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.245639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.245813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.245941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.246077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.246204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.246330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.246455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.246580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.246704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.246837 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.246964 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.247089 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.247213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.247343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.247468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.247610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.247736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.247857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.247983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.248106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.248106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.248357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.248480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.248561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.248724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.248848 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.248972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.249106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.249240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.249366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.249490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.249614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.249752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.249882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.250011 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.250135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.250256 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.250378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.250499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.250628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.250751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.250873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.251004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.251126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.251126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.251363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.251486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.251607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.251728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.251849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.251970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.252099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.252224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.252346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.252468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.252599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.252721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.252850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.252972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.253094 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.253215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.253338 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.253459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.253588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.253762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.253884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.254011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.254135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.254259 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.254405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.254405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.254640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.254764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.254889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.255013 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.255144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.255269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.255396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.255521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.255646 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.255770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.255902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.256026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.256159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.256283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.256409 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.256532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.256663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.256787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.256910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.257034 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.257158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.257280 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.257419 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.257540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.257665 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.257818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.257944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.258070 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.258205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.258332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.258459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.258584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.258707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.258792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.258873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.258951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259024 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259094 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] resource.caching = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.259962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] resource.caching = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.260899 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] role.caching = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] role.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261402 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] role.caching = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] role.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.261965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262694 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262843 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.262987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.263057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.263127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.263197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.263276 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.263349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.263420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.263617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.263719 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.263864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.263945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.264198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.264283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.264454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.264533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.264705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.264785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.264953 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.265031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.265189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.265268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.265433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.265512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.265666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.265765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.265942 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.266021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.266176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.266256 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.266411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.266490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.266640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.266717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.266874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.266955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.267106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.267185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.267355 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.267433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.267594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.267673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.267831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.267910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.268062 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.268141 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.268293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.268372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] token.caching = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.268523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.268601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.268759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] token.caching = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.268843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.268992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.269072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.269240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.269319 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.269474 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.269552 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.269738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.269862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.269985 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.270076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.270230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.270309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.270478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.270559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.270717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.270795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.270956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.271034 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.271192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.271272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.271425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.271503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.271657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.271735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.271901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.271982 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.272153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.272240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.272393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.272479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.272635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.272715 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.connection = **** {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.272899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.272985 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.273146 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.273725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.connection = **** {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.273919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.273998 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.274161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.274252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.274396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.274474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.274632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.274709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.274863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.274942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.275097 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.275175 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.275328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.275405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.275561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.275642 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.275804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.275883 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.276037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.276115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.276281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.276368 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.276515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.276594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.276757 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.276834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.276989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.277068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.277227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.277305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.277455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.277533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.277613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87945) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 08:31:21.277732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.277934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 08:31:21.278104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87946) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 08:31:21.307187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.307420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.307651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.307851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.308085 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.308312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.316019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.316973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.344322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.344555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.345772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.346003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.397953 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.398138 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.446537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.446773 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.447247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.447483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.601185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.601441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.601583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.601668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.601769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.601914 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.602043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.602133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.602210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.602329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.602438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.602804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.625225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.625398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.625485 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.625632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.625751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.625884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.625968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.626100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.626184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.626313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.626398 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.626522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.626605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.626731 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.638068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.638335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.650799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.650967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.651190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.651415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.653312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.653924 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.654007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.654072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.654155 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.654222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87945) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.654297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.654297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.654297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.654297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87946) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 08:31:21.669147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.669359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 08:31:21.669556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.669730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.669835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 08:31:21.669941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 08:31:21.670129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.670212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.670331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 08:31:21.670469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.670605 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.670730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.670867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.670990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.671127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.671385 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.681213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.681483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.681568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.681804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.681889 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.682098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.682184 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.682380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.689081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.689338 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.692026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.692248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.692331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.692534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.692616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.692817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.692912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.693103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.693187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.693388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.693470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.693650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.693754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.693946 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.694029 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.694216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.694308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.694491 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.694571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.694762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.694842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.695060 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.717558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.717917 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.719047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.719338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.719420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.719644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.724027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.724317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.724399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.724603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.724697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.724882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.731823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.732042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.732130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.732327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.737792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.737964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.738089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.738259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.738378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.738548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.738666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.738835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.738947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.739111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.739227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.739385 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.739501 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.739653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.739773 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.739918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.755898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.756179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.756330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.756612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.760099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.760534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.763383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.763685 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.763767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.764067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.767927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.768297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.771178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.771535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.774466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.774771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.774856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.775059 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.775143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.775346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.775437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.775631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.775714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.775911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.775994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.776200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.792341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.792598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.792681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.792887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.793020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.793149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.793231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.793410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.800046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.800629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.801602 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.802202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.803157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.803449 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.803740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.804018 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.806333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.806632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.806830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.806914 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.807115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.807210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.807397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.807561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.807681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.808074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.818772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.819064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.819290 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.819373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.819578 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.819661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.819860 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.820000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.820674 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.820826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.821056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.821136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.821336 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.821616 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.836669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.837105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.839636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.840042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.842547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.843234 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.844401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.844717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.846044 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.846220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.846315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.846502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.846720 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.846900 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.853946 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.854348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.855409 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.855804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.858333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.858716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.858800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.859009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.859103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.859294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.859378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.859651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.867259 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.867540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.867636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.867824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.867906 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.868102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.874791 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87945) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.875017 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87946) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 08:31:21.875103 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.875301 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.875384 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.875586 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.875669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.875865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.875948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.876142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.876225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.876414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.876555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.876685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.876779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.876954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.877044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.877232 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.877313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.877499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.877582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.877835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.877927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.878096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.878178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.878361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.878443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87945) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.878637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87946) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 08:31:21.913025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563078c359c0 pid: 87945 (default app) Oct 04 08:31:21.913419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563078c359c0 pid: 87946 (default app) Oct 04 08:31:21.916332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f89273-e4f6-4a7e-9773-7d1d20536704 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:21.916625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f89273-e4f6-4a7e-9773-7d1d20536704 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:21.916899 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f89273-e4f6-4a7e-9773-7d1d20536704 None None] PATH_INFO: `/v3` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:21.921067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Tue Oct 4 08:31:21 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:28.796629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d17170-169a-4de1-a816-98b0fd70866c None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:28.797163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d17170-169a-4de1-a816-98b0fd70866c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:28.797163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d17170-169a-4de1-a816-98b0fd70866c None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:28.801480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:31:28.807300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3febc2b-727f-4318-ad33-7dce47ed9227 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:28.807819 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3febc2b-727f-4318-ad33-7dce47ed9227 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:28.807915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3febc2b-727f-4318-ad33-7dce47ed9227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:28.878053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: INFO dbcounter [None req-d3febc2b-727f-4318-ad33-7dce47ed9227 None None] Registered counter for database keystone Oct 04 08:31:28.897217 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG oslo_db.sqlalchemy.engines [None req-d3febc2b-727f-4318-ad33-7dce47ed9227 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87945) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:31:28.903480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writer thread running {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:31:29.079838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG passlib.handlers.bcrypt [None req-d3febc2b-727f-4318-ad33-7dce47ed9227 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87945) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 08:31:29.080219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG passlib.handlers.bcrypt [None req-d3febc2b-727f-4318-ad33-7dce47ed9227 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87945) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 08:31:29.138114 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d3febc2b-727f-4318-ad33-7dce47ed9227 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:29.139725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d3febc2b-727f-4318-ad33-7dce47ed9227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:29.268127 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 04 08:31:29.680137 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d17170-169a-4de1-a816-98b0fd70866c None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:29.749101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: INFO dbcounter [None req-02d17170-169a-4de1-a816-98b0fd70866c None None] Registered counter for database keystone Oct 04 08:31:29.770041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG oslo_db.sqlalchemy.engines [None req-02d17170-169a-4de1-a816-98b0fd70866c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87946) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 08:31:29.776991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writer thread running {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 08:31:29.893949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d17170-169a-4de1-a816-98b0fd70866c None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:29.898242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d564ce-7200-4ff7-bf3f-0f6a1ffbee3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:29.899431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d564ce-7200-4ff7-bf3f-0f6a1ffbee3d admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:29.899715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d564ce-7200-4ff7-bf3f-0f6a1ffbee3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:29.899985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d564ce-7200-4ff7-bf3f-0f6a1ffbee3d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:29.908128 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d564ce-7200-4ff7-bf3f-0f6a1ffbee3d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:29.959687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: WARNING keystone.common.rbac_enforcer.enforcer [None req-38d564ce-7200-4ff7-bf3f-0f6a1ffbee3d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 08:31:29.960028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d564ce-7200-4ff7-bf3f-0f6a1ffbee3d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:29.969288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Tue Oct 4 08:31:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:31:29.975332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3febc2b-727f-4318-ad33-7dce47ed9227 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:29.993281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3febc2b-727f-4318-ad33-7dce47ed9227 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:29.997650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb93e43-703f-4a7a-8778-d030cd1a1b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:29.998768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb93e43-703f-4a7a-8778-d030cd1a1b5a admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:29.999052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb93e43-703f-4a7a-8778-d030cd1a1b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:29.999322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb93e43-703f-4a7a-8778-d030cd1a1b5a admin admin] PATH_INFO: `/v3/projects/47b3fa8678ad4f779e78d8c72364fac2` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:30.000904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb93e43-703f-4a7a-8778-d030cd1a1b5a admin admin] RBAC: Authorizing `identity:get_project(project_id=47b3fa8678ad4f779e78d8c72364fac2)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:30.055890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ceb93e43-703f-4a7a-8778-d030cd1a1b5a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 08:31:30.056244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb93e43-703f-4a7a-8778-d030cd1a1b5a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:30.060809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Tue Oct 4 08:31:29 2022] GET /identity/v3/projects/47b3fa8678ad4f779e78d8c72364fac2 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:32.154817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4082d-c591-4328-81ed-64032e0ffcc0 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:32.155331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4082d-c591-4328-81ed-64032e0ffcc0 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:32.155436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4082d-c591-4328-81ed-64032e0ffcc0 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:32.159462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:32.165449 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35c0c6-fc4e-46de-8e29-23d394744d28 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:32.165830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35c0c6-fc4e-46de-8e29-23d394744d28 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:32.166025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35c0c6-fc4e-46de-8e29-23d394744d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:32.186289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-cf35c0c6-fc4e-46de-8e29-23d394744d28 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:32.187818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-cf35c0c6-fc4e-46de-8e29-23d394744d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:32.195566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 04 08:31:32.460352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4082d-c591-4328-81ed-64032e0ffcc0 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:32.472862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4082d-c591-4328-81ed-64032e0ffcc0 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:32.477642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f3378-40fd-454f-af6d-11eefa252d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:32.478532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f3378-40fd-454f-af6d-11eefa252d10 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:32.478814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f3378-40fd-454f-af6d-11eefa252d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:32.479087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f3378-40fd-454f-af6d-11eefa252d10 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:32.486605 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f3378-40fd-454f-af6d-11eefa252d10 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:32.488378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f3378-40fd-454f-af6d-11eefa252d10 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:32.497004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Tue Oct 4 08:31:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 08:31:32.502195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35c0c6-fc4e-46de-8e29-23d394744d28 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:32.511142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35c0c6-fc4e-46de-8e29-23d394744d28 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:32.515387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3351993-334f-422d-b1dd-f330d4bb05a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:32.515999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3351993-334f-422d-b1dd-f330d4bb05a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:32.516278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3351993-334f-422d-b1dd-f330d4bb05a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:32.516550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3351993-334f-422d-b1dd-f330d4bb05a9 admin admin] PATH_INFO: `/v3/users/f84b5144054d4e158a838b4ae551996f` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:32.517877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3351993-334f-422d-b1dd-f330d4bb05a9 admin admin] RBAC: Authorizing `identity:get_user(user_id=f84b5144054d4e158a838b4ae551996f)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:32.519689 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3351993-334f-422d-b1dd-f330d4bb05a9 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:32.521209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Tue Oct 4 08:31:32 2022] GET /identity/v3/users/f84b5144054d4e158a838b4ae551996f => generated 247 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:34.703185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749634e0-128b-4be0-b8fc-e8b59d3aee17 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:34.703739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749634e0-128b-4be0-b8fc-e8b59d3aee17 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:34.703739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749634e0-128b-4be0-b8fc-e8b59d3aee17 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:34.705116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:34.710454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf232b-6eb6-49c2-8dd5-4b19527f3c1e None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:34.711400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf232b-6eb6-49c2-8dd5-4b19527f3c1e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:34.711520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf232b-6eb6-49c2-8dd5-4b19527f3c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:34.730140 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-0caf232b-6eb6-49c2-8dd5-4b19527f3c1e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:34.731329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-0caf232b-6eb6-49c2-8dd5-4b19527f3c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:34.752211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a83fe-d616-4af6-8f91-e8966a7b3446 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:34.752493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a83fe-d616-4af6-8f91-e8966a7b3446 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:34.752764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a83fe-d616-4af6-8f91-e8966a7b3446 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:34.754102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:31:34.759096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d35599-c7c8-48ab-946c-2bfc6e9ffae4 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:34.759381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d35599-c7c8-48ab-946c-2bfc6e9ffae4 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:34.759658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d35599-c7c8-48ab-946c-2bfc6e9ffae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:34.801503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG passlib.handlers.bcrypt [None req-40d35599-c7c8-48ab-946c-2bfc6e9ffae4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87946) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 08:31:34.801928 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG passlib.handlers.bcrypt [None req-40d35599-c7c8-48ab-946c-2bfc6e9ffae4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87946) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 08:31:34.812232 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:31:34.850790 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-40d35599-c7c8-48ab-946c-2bfc6e9ffae4 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:34.853230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-40d35599-c7c8-48ab-946c-2bfc6e9ffae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:34.950235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:31:35.079935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf232b-6eb6-49c2-8dd5-4b19527f3c1e None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:35.105913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:35.184531 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf232b-6eb6-49c2-8dd5-4b19527f3c1e None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:35.214846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d35599-c7c8-48ab-946c-2bfc6e9ffae4 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:35.218151 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ffb681-b88c-450e-8da4-284ef8c86108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:35.218797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffb681-b88c-450e-8da4-284ef8c86108 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:35.219070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffb681-b88c-450e-8da4-284ef8c86108 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:35.219337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffb681-b88c-450e-8da4-284ef8c86108 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:35.230816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ffb681-b88c-450e-8da4-284ef8c86108 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-53ffb681-b88c-450e-8da4-284ef8c86108 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:35.239540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:35.240483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:31:35.241590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:31:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:31:35.246317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ffb681-b88c-450e-8da4-284ef8c86108 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:35.317055 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d35599-c7c8-48ab-946c-2bfc6e9ffae4 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:35.319283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:35.327216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:35.345943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ffb681-b88c-450e-8da4-284ef8c86108 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:35.355816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357c96d-f9ac-4892-b87f-20652938f42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:35.356683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357c96d-f9ac-4892-b87f-20652938f42f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:35.356960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357c96d-f9ac-4892-b87f-20652938f42f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:35.357228 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357c96d-f9ac-4892-b87f-20652938f42f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:35.379814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbada7fa-eb0a-4fd6-9812-3065688006d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:35.380416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbada7fa-eb0a-4fd6-9812-3065688006d5 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:35.380695 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbada7fa-eb0a-4fd6-9812-3065688006d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:35.380968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbada7fa-eb0a-4fd6-9812-3065688006d5 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:35.381639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbada7fa-eb0a-4fd6-9812-3065688006d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:35.381967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbada7fa-eb0a-4fd6-9812-3065688006d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:35.384175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbada7fa-eb0a-4fd6-9812-3065688006d5 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:35.395521 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:31:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:35.401801 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbada7fa-eb0a-4fd6-9812-3065688006d5 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:35.423801 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357c96d-f9ac-4892-b87f-20652938f42f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:35.425312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357c96d-f9ac-4892-b87f-20652938f42f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:35.500370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbada7fa-eb0a-4fd6-9812-3065688006d5 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:35.523187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:31:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:31:35.528873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357c96d-f9ac-4892-b87f-20652938f42f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:35.531547 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:35.533419 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c5a4c-b64d-4d0d-8672-5d0bb2a6f792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:35.534074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c5a4c-b64d-4d0d-8672-5d0bb2a6f792 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:35.534355 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c5a4c-b64d-4d0d-8672-5d0bb2a6f792 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:35.534626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c5a4c-b64d-4d0d-8672-5d0bb2a6f792 None admin] PATH_INFO: `/v3/users/f84b5144054d4e158a838b4ae551996f` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:35.535761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c5a4c-b64d-4d0d-8672-5d0bb2a6f792 None admin] RBAC: Authorizing `identity:get_user(user_id=f84b5144054d4e158a838b4ae551996f)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:35.537213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c5a4c-b64d-4d0d-8672-5d0bb2a6f792 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:35.538768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:31:35 2022] GET /identity/v3/users/f84b5144054d4e158a838b4ae551996f => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:35.544032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c5a4c-b64d-4d0d-8672-5d0bb2a6f792 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:35.558756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:35.627951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357c96d-f9ac-4892-b87f-20652938f42f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:35.640846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c5a4c-b64d-4d0d-8672-5d0bb2a6f792 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:35.662927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3fb54-2844-49c3-b7a7-a666b0ceb8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:35.663786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3fb54-2844-49c3-b7a7-a666b0ceb8e3 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:35.664631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3fb54-2844-49c3-b7a7-a666b0ceb8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:35.664912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3fb54-2844-49c3-b7a7-a666b0ceb8e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:35.675112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40385d01-4cd3-4d59-bf06-418b4c5b7df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:35.675741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40385d01-4cd3-4d59-bf06-418b4c5b7df6 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:35.676020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40385d01-4cd3-4d59-bf06-418b4c5b7df6 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:35.676293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40385d01-4cd3-4d59-bf06-418b4c5b7df6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:35.677402 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40385d01-4cd3-4d59-bf06-418b4c5b7df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:35.678887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40385d01-4cd3-4d59-bf06-418b4c5b7df6 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:35.680441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 10/18] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:31:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:35.683548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3fb54-2844-49c3-b7a7-a666b0ceb8e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:35.685017 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3fb54-2844-49c3-b7a7-a666b0ceb8e3 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:35.686183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40385d01-4cd3-4d59-bf06-418b4c5b7df6 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-78d3fb54-2844-49c3-b7a7-a666b0ceb8e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:31:35.703165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:35.704646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:31:35.705566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:31:35.705566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:31:35.705566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:31:35.705566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 9/19] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:31:35.709909 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3fb54-2844-49c3-b7a7-a666b0ceb8e3 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:35.765329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:35.769747 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:35.789421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40385d01-4cd3-4d59-bf06-418b4c5b7df6 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:35.810319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3fb54-2844-49c3-b7a7-a666b0ceb8e3 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:35.821894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ea17a-ed9f-441c-8d7e-5657d476455a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:35.822499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ea17a-ed9f-441c-8d7e-5657d476455a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:35.822778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ea17a-ed9f-441c-8d7e-5657d476455a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:35.823048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ea17a-ed9f-441c-8d7e-5657d476455a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:35.823762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ea17a-ed9f-441c-8d7e-5657d476455a None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.domain.id=default, user.id=f84b5144054d4e158a838b4ae551996f) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:35.824066 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ea17a-ed9f-441c-8d7e-5657d476455a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:35.825503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ea17a-ed9f-441c-8d7e-5657d476455a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:35.831848 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Tue Oct 4 08:31:35 2022] GET /identity/v3/role_assignments?user.id=f84b5144054d4e158a838b4ae551996f&scope.domain.id=default&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:35.842707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec2149-fc2e-4911-a4f5-8fd3571c7b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:35.843351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec2149-fc2e-4911-a4f5-8fd3571c7b05 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:35.843635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec2149-fc2e-4911-a4f5-8fd3571c7b05 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:35.843909 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec2149-fc2e-4911-a4f5-8fd3571c7b05 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:35.844576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec2149-fc2e-4911-a4f5-8fd3571c7b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:35.844884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec2149-fc2e-4911-a4f5-8fd3571c7b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:35.846368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec2149-fc2e-4911-a4f5-8fd3571c7b05 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:35.866648 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:38.324180 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d19c5-f33f-4d48-bd46-ef0ab176ec93 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:38.324726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d19c5-f33f-4d48-bd46-ef0ab176ec93 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:38.324726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d19c5-f33f-4d48-bd46-ef0ab176ec93 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:38.326114 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:38.331779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a3b4a-5e9a-4cb8-aef7-297c24a3d92e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:38.332294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a3b4a-5e9a-4cb8-aef7-297c24a3d92e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:38.332369 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a3b4a-5e9a-4cb8-aef7-297c24a3d92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:38.359185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-ba1a3b4a-5e9a-4cb8-aef7-297c24a3d92e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:38.360388 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ba1a3b4a-5e9a-4cb8-aef7-297c24a3d92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:38.441723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ff687-2cc0-4073-b37f-af61052e55d8 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:38.442061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ff687-2cc0-4073-b37f-af61052e55d8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:38.442276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ff687-2cc0-4073-b37f-af61052e55d8 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:38.443583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 13/23] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:31:38.450736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb5c5f-a62c-4121-a901-710e3944d26c None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:38.451017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb5c5f-a62c-4121-a901-710e3944d26c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:38.451287 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb5c5f-a62c-4121-a901-710e3944d26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:38.469366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 11/24] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:38.478917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-89eb5c5f-a62c-4121-a901-710e3944d26c None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:38.480125 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-89eb5c5f-a62c-4121-a901-710e3944d26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:38.569615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 14/25] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:31:38.806281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a50e3-eed8-4261-9d2b-83e1fe5a8f93 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:38.806929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a50e3-eed8-4261-9d2b-83e1fe5a8f93 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:38.807393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a50e3-eed8-4261-9d2b-83e1fe5a8f93 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:38.808916 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 12/26] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:38.813818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0e25a-4af8-4d63-a367-3ded52e4bc08 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:38.814326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0e25a-4af8-4d63-a367-3ded52e4bc08 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:38.814787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0e25a-4af8-4d63-a367-3ded52e4bc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:38.818249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb5c5f-a62c-4121-a901-710e3944d26c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:38.851189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-3fe0e25a-4af8-4d63-a367-3ded52e4bc08 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:38.854245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3fe0e25a-4af8-4d63-a367-3ded52e4bc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:38.954418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:38.986431 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:38.991072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 13/27] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:31:38.992453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c5618-6b8a-4d79-a2c8-8047321917e7 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:38.992749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c5618-6b8a-4d79-a2c8-8047321917e7 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:38.993014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c5618-6b8a-4d79-a2c8-8047321917e7 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:38.994496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 14/28] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:39.008730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd95be4-9f15-4b34-9099-e96d834ef50d None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:39.009014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd95be4-9f15-4b34-9099-e96d834ef50d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:39.009359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd95be4-9f15-4b34-9099-e96d834ef50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:39.036694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb5c5f-a62c-4121-a901-710e3944d26c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:39.038221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-6fd95be4-9f15-4b34-9099-e96d834ef50d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:39.039493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6fd95be4-9f15-4b34-9099-e96d834ef50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:39.108656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a600b-9a12-42d5-a4d2-fb2a34c57b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:39.109301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a600b-9a12-42d5-a4d2-fb2a34c57b75 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:39.109586 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a600b-9a12-42d5-a4d2-fb2a34c57b75 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:39.109890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a600b-9a12-42d5-a4d2-fb2a34c57b75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:39.120294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 15/29] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:31:39.121266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd95be4-9f15-4b34-9099-e96d834ef50d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:39.130809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a600b-9a12-42d5-a4d2-fb2a34c57b75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-477a600b-9a12-42d5-a4d2-fb2a34c57b75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:39.136342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:39.137254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:31:39.138094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 15/30] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:31:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:31:39.143513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a600b-9a12-42d5-a4d2-fb2a34c57b75 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:39.229123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd95be4-9f15-4b34-9099-e96d834ef50d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:39.237568 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:39.267751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea309f8-7d89-4493-9554-011cc4481ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:39.268639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea309f8-7d89-4493-9554-011cc4481ed4 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:39.269095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea309f8-7d89-4493-9554-011cc4481ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:39.269535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea309f8-7d89-4493-9554-011cc4481ed4 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:39.270529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea309f8-7d89-4493-9554-011cc4481ed4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:39.272139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea309f8-7d89-4493-9554-011cc4481ed4 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:39.360314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a600b-9a12-42d5-a4d2-fb2a34c57b75 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:39.410167 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 16/31] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Oct 4 08:31:39 2022] POST /identity/v3/roles => generated 219 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:31:39.412458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea309f8-7d89-4493-9554-011cc4481ed4 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:39.446454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0195da7-a91b-4511-8ea5-765e5ea93c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:39.447523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0195da7-a91b-4511-8ea5-765e5ea93c7e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:39.447992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0195da7-a91b-4511-8ea5-765e5ea93c7e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:39.448466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0195da7-a91b-4511-8ea5-765e5ea93c7e None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:39.449327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0195da7-a91b-4511-8ea5-765e5ea93c7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:39.449839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0195da7-a91b-4511-8ea5-765e5ea93c7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:39.451450 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0195da7-a91b-4511-8ea5-765e5ea93c7e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:39.461882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:39.479299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 16/32] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:31:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:39.483692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0195da7-a91b-4511-8ea5-765e5ea93c7e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:39.642478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea309f8-7d89-4493-9554-011cc4481ed4 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:39.647368 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0195da7-a91b-4511-8ea5-765e5ea93c7e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:39.705533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a10a2-91d0-4646-838f-92e1dd37a379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:39.706291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a10a2-91d0-4646-838f-92e1dd37a379 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:39.706577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a10a2-91d0-4646-838f-92e1dd37a379 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:39.706880 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a10a2-91d0-4646-838f-92e1dd37a379 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:39.707630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a10a2-91d0-4646-838f-92e1dd37a379 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:39.709164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a10a2-91d0-4646-838f-92e1dd37a379 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:39.712824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059a739-eed5-4189-89e3-07d9bee7bedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:39.713471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a739-eed5-4189-89e3-07d9bee7bedd None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:39.713798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a739-eed5-4189-89e3-07d9bee7bedd None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:39.714078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059a739-eed5-4189-89e3-07d9bee7bedd None admin] PATH_INFO: `/v3/users/f84b5144054d4e158a838b4ae551996f` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:39.715280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059a739-eed5-4189-89e3-07d9bee7bedd None admin] RBAC: Authorizing `identity:get_user(user_id=f84b5144054d4e158a838b4ae551996f)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:39.717244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059a739-eed5-4189-89e3-07d9bee7bedd None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:39.720772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 17/33] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:31:39 2022] GET /identity/v3/users/f84b5144054d4e158a838b4ae551996f => generated 247 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:39.720772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059a739-eed5-4189-89e3-07d9bee7bedd None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:39.731695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 17/34] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Oct 4 08:31:39 2022] POST /identity/v3/roles => generated 225 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:31:39.733436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a10a2-91d0-4646-838f-92e1dd37a379 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:39.790099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:39.801476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:39.860242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059a739-eed5-4189-89e3-07d9bee7bedd None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:39.893317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a10a2-91d0-4646-838f-92e1dd37a379 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:39.939723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8ee5a-abbf-48a1-ae34-649d3448f720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:39.940392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8ee5a-abbf-48a1-ae34-649d3448f720 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:39.940678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8ee5a-abbf-48a1-ae34-649d3448f720 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:39.940960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8ee5a-abbf-48a1-ae34-649d3448f720 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:39.943742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8ee5a-abbf-48a1-ae34-649d3448f720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:39.946293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8ee5a-abbf-48a1-ae34-649d3448f720 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:39.948001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 18/35] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:31:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:39.955487 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8ee5a-abbf-48a1-ae34-649d3448f720 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:39.956794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82421b-0dbe-4fd8-a7ee-c39ee5146cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:39.957459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82421b-0dbe-4fd8-a7ee-c39ee5146cc5 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:39.957800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82421b-0dbe-4fd8-a7ee-c39ee5146cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:39.958078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82421b-0dbe-4fd8-a7ee-c39ee5146cc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:39.979311 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82421b-0dbe-4fd8-a7ee-c39ee5146cc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:39.981284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82421b-0dbe-4fd8-a7ee-c39ee5146cc5 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-9d82421b-0dbe-4fd8-a7ee-c39ee5146cc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:31:39.999897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:40.000749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:31:40.002203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:31:40.002203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:31:40.002203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:31:40.002203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 18/36] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:31:40.006932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82421b-0dbe-4fd8-a7ee-c39ee5146cc5 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:40.096629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:40.107448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8ee5a-abbf-48a1-ae34-649d3448f720 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:40.152831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82421b-0dbe-4fd8-a7ee-c39ee5146cc5 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:40.157860 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602fd15-fda7-42e9-b129-919cfe7ad01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:40.158964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602fd15-fda7-42e9-b129-919cfe7ad01b None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:40.159446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602fd15-fda7-42e9-b129-919cfe7ad01b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:40.159930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602fd15-fda7-42e9-b129-919cfe7ad01b None admin] PATH_INFO: `/v3/domains/default/users/f84b5144054d4e158a838b4ae551996f/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:40.164106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602fd15-fda7-42e9-b129-919cfe7ad01b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f84b5144054d4e158a838b4ae551996f, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:40.167119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602fd15-fda7-42e9-b129-919cfe7ad01b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:40.173819 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:40.191188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 19/37] 10.222.0.26 () {58 vars in 1361 bytes} [Tue Oct 4 08:31:39 2022] PUT /identity/v3/domains/default/users/f84b5144054d4e158a838b4ae551996f/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:31:40.198562 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1056bf3-439f-46cd-9d9c-ddb519253d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:40.199185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1056bf3-439f-46cd-9d9c-ddb519253d58 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:40.199465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1056bf3-439f-46cd-9d9c-ddb519253d58 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:40.199738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1056bf3-439f-46cd-9d9c-ddb519253d58 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:40.200398 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1056bf3-439f-46cd-9d9c-ddb519253d58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:40.200717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1056bf3-439f-46cd-9d9c-ddb519253d58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:40.202194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1056bf3-439f-46cd-9d9c-ddb519253d58 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:40.224065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 19/38] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:40.226996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d1cf8-5e67-482d-84af-16c600ac30ce None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:40.227288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d1cf8-5e67-482d-84af-16c600ac30ce None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:40.227569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d1cf8-5e67-482d-84af-16c600ac30ce None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:40.229450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 20/39] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:40.229908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1056bf3-439f-46cd-9d9c-ddb519253d58 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:40.235924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d6ad4-7850-4098-8711-f1838dce81a7 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:40.236251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d6ad4-7850-4098-8711-f1838dce81a7 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:40.236491 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d6ad4-7850-4098-8711-f1838dce81a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:40.259407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d17d6ad4-7850-4098-8711-f1838dce81a7 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:40.261214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d17d6ad4-7850-4098-8711-f1838dce81a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:40.340177 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 21/40] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:31:40.341475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7326f36-173b-497c-a1c4-3ec067c9e59e None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:40.341841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7326f36-173b-497c-a1c4-3ec067c9e59e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:40.342092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7326f36-173b-497c-a1c4-3ec067c9e59e None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:40.343662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 22/41] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:40.349262 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9001a4-e779-4176-b06a-040ef14b288e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:40.349602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9001a4-e779-4176-b06a-040ef14b288e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:40.349918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9001a4-e779-4176-b06a-040ef14b288e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:40.360409 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1056bf3-439f-46cd-9d9c-ddb519253d58 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:40.371336 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-2d9001a4-e779-4176-b06a-040ef14b288e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:40.372639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2d9001a4-e779-4176-b06a-040ef14b288e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:40.403911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5bacf-a61e-413d-b236-1b11a89eeb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:40.404572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5bacf-a61e-413d-b236-1b11a89eeb88 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:40.404848 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5bacf-a61e-413d-b236-1b11a89eeb88 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:40.405117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5bacf-a61e-413d-b236-1b11a89eeb88 None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:40.405814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5bacf-a61e-413d-b236-1b11a89eeb88 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:40.407427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5bacf-a61e-413d-b236-1b11a89eeb88 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:40.457348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 23/42] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:31:40.626195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9001a4-e779-4176-b06a-040ef14b288e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:40.642356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:40.652262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 20/43] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:31:40 2022] POST /identity/v3/projects => generated 301 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:31:40.730570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5bacf-a61e-413d-b236-1b11a89eeb88 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:40.763605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:40.774189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9001a4-e779-4176-b06a-040ef14b288e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:40.825724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed68cf2-956f-4c73-b741-5bbcda12412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:40.826325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed68cf2-956f-4c73-b741-5bbcda12412e None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:40.826610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed68cf2-956f-4c73-b741-5bbcda12412e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:40.826996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed68cf2-956f-4c73-b741-5bbcda12412e None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:40.827698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed68cf2-956f-4c73-b741-5bbcda12412e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:40.829193 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed68cf2-956f-4c73-b741-5bbcda12412e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:40.847194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 24/44] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Oct 4 08:31:40 2022] POST /identity/v3/roles => generated 223 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:31:40.851671 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5bacf-a61e-413d-b236-1b11a89eeb88 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:40.902303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b3140-ade9-470f-8831-514c0829a4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:40.902880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b3140-ade9-470f-8831-514c0829a4cc None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:40.903280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b3140-ade9-470f-8831-514c0829a4cc None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:40.903507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b3140-ade9-470f-8831-514c0829a4cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:40.904912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b3140-ade9-470f-8831-514c0829a4cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:40.906739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9b3140-ade9-470f-8831-514c0829a4cc None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:40.908605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 21/45] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:31:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:40.914602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed68cf2-956f-4c73-b741-5bbcda12412e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:41.004938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:41.037230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed68cf2-956f-4c73-b741-5bbcda12412e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:41.075162 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc2d2c-b433-42cd-995a-41fc0eb7dc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:41.075788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2d2c-b433-42cd-995a-41fc0eb7dc27 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:41.076068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2d2c-b433-42cd-995a-41fc0eb7dc27 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:41.076340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2d2c-b433-42cd-995a-41fc0eb7dc27 None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:41.076988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc2d2c-b433-42cd-995a-41fc0eb7dc27 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:41.078483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc2d2c-b433-42cd-995a-41fc0eb7dc27 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:41.108895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 25/46] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:31:40 2022] POST /identity/v3/projects => generated 312 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:31:42.615708 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01987141-b925-4295-8aa8-3dfa0af1d7bb None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:42.616251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01987141-b925-4295-8aa8-3dfa0af1d7bb None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:42.616251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01987141-b925-4295-8aa8-3dfa0af1d7bb None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:42.617603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 22/47] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:42.622866 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e332a3-88f1-49cd-9b54-e3ed8cdea342 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:42.623265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e332a3-88f1-49cd-9b54-e3ed8cdea342 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:42.623431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e332a3-88f1-49cd-9b54-e3ed8cdea342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:42.641664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-29e332a3-88f1-49cd-9b54-e3ed8cdea342 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:42.642870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-29e332a3-88f1-49cd-9b54-e3ed8cdea342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:42.708714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 26/48] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:42.978027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01987141-b925-4295-8aa8-3dfa0af1d7bb None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:43.071174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:43.078734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01987141-b925-4295-8aa8-3dfa0af1d7bb None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:43.112281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d36f6-bf0c-4eb3-baef-94e739cc8a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:43.112910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d36f6-bf0c-4eb3-baef-94e739cc8a15 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:43.113194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d36f6-bf0c-4eb3-baef-94e739cc8a15 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:43.113469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d36f6-bf0c-4eb3-baef-94e739cc8a15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:43.124335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d36f6-bf0c-4eb3-baef-94e739cc8a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-dc3d36f6-bf0c-4eb3-baef-94e739cc8a15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:43.129367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:43.130401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:31:43.131147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 23/49] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:31:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:31:43.137517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e332a3-88f1-49cd-9b54-e3ed8cdea342 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:43.197706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:43.243961 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e332a3-88f1-49cd-9b54-e3ed8cdea342 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:43.251324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c913c1-cade-491d-9219-b2b836b47160 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:43.251610 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c913c1-cade-491d-9219-b2b836b47160 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:43.251880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c913c1-cade-491d-9219-b2b836b47160 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:43.253212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 24/50] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:31:43.258707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079a191-4e0c-4064-a81c-9740e9d52d29 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:43.258990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079a191-4e0c-4064-a81c-9740e9d52d29 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:43.259261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079a191-4e0c-4064-a81c-9740e9d52d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:43.277018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d079a191-4e0c-4064-a81c-9740e9d52d29 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:43.277296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4292906-e942-46d1-9439-ed073be29d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:43.277943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4292906-e942-46d1-9439-ed073be29d56 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:43.278227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4292906-e942-46d1-9439-ed073be29d56 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:43.278501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4292906-e942-46d1-9439-ed073be29d56 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:43.278587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d079a191-4e0c-4064-a81c-9740e9d52d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:43.279165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4292906-e942-46d1-9439-ed073be29d56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:43.279471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4292906-e942-46d1-9439-ed073be29d56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:43.280851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4292906-e942-46d1-9439-ed073be29d56 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:43.293787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 27/51] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:31:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:43.299393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4292906-e942-46d1-9439-ed073be29d56 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:43.347936 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 25/52] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:31:43.498746 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4292906-e942-46d1-9439-ed073be29d56 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:43.515258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:43.532065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df19841-f85b-459c-be4e-dd23c464726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:43.532703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df19841-f85b-459c-be4e-dd23c464726c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:43.532986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df19841-f85b-459c-be4e-dd23c464726c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:43.533263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df19841-f85b-459c-be4e-dd23c464726c None admin] PATH_INFO: `/v3/users/f84b5144054d4e158a838b4ae551996f` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:43.534426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df19841-f85b-459c-be4e-dd23c464726c None admin] RBAC: Authorizing `identity:get_user(user_id=f84b5144054d4e158a838b4ae551996f)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:43.535881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df19841-f85b-459c-be4e-dd23c464726c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:43.537594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 28/53] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:31:43 2022] GET /identity/v3/users/f84b5144054d4e158a838b4ae551996f => generated 247 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:43.543421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079a191-4e0c-4064-a81c-9740e9d52d29 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:43.618537 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df19841-f85b-459c-be4e-dd23c464726c None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:43.624877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:43.640392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079a191-4e0c-4064-a81c-9740e9d52d29 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:43.674221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b08deb-2185-4abd-a2d1-dc2f0ad81836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:43.674853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b08deb-2185-4abd-a2d1-dc2f0ad81836 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:43.675138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b08deb-2185-4abd-a2d1-dc2f0ad81836 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:43.675415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b08deb-2185-4abd-a2d1-dc2f0ad81836 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:43.676514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b08deb-2185-4abd-a2d1-dc2f0ad81836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:43.678023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b08deb-2185-4abd-a2d1-dc2f0ad81836 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:43.679601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 26/54] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:31:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:43.685572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b08deb-2185-4abd-a2d1-dc2f0ad81836 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:43.717709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df19841-f85b-459c-be4e-dd23c464726c None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:43.750744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6af2b-72ac-451a-88e6-45de5a81862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:43.751315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6af2b-72ac-451a-88e6-45de5a81862e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:43.751597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6af2b-72ac-451a-88e6-45de5a81862e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:43.751871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6af2b-72ac-451a-88e6-45de5a81862e None admin] PATH_INFO: `/v3/domains/default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:43.753030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6af2b-72ac-451a-88e6-45de5a81862e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:43.754597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6af2b-72ac-451a-88e6-45de5a81862e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:43.756227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 29/55] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:31:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:43.762263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6af2b-72ac-451a-88e6-45de5a81862e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:43.783575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b08deb-2185-4abd-a2d1-dc2f0ad81836 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:43.815482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:43.817247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070534b0-83ed-49d3-aa29-5e9f6fbf45e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:43.817876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070534b0-83ed-49d3-aa29-5e9f6fbf45e5 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:43.818201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070534b0-83ed-49d3-aa29-5e9f6fbf45e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:43.818477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070534b0-83ed-49d3-aa29-5e9f6fbf45e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:43.819211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070534b0-83ed-49d3-aa29-5e9f6fbf45e5 None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.domain.id=default, user.id=f84b5144054d4e158a838b4ae551996f) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:43.819535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070534b0-83ed-49d3-aa29-5e9f6fbf45e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:43.821545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070534b0-83ed-49d3-aa29-5e9f6fbf45e5 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:43.827632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 27/56] 10.222.0.26 () {56 vars in 1322 bytes} [Tue Oct 4 08:31:43 2022] GET /identity/v3/role_assignments?user.id=f84b5144054d4e158a838b4ae551996f&scope.domain.id=default&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:43.864198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6af2b-72ac-451a-88e6-45de5a81862e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:43.900507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab94d9d-ef31-4592-9895-3bfca19083fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:43.901136 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab94d9d-ef31-4592-9895-3bfca19083fc None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:43.901418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab94d9d-ef31-4592-9895-3bfca19083fc None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:43.901724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab94d9d-ef31-4592-9895-3bfca19083fc None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:43.902389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab94d9d-ef31-4592-9895-3bfca19083fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:43.903859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab94d9d-ef31-4592-9895-3bfca19083fc None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:43.932123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 30/57] 10.222.0.26 () {60 vars in 1140 bytes} [Tue Oct 4 08:31:43 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:31:46.061103 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae6188-2f09-4abe-8b79-8a294df564af None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:46.061591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae6188-2f09-4abe-8b79-8a294df564af None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:46.061674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae6188-2f09-4abe-8b79-8a294df564af None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:46.063012 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 28/58] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:46.068233 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd338b-d740-4bc1-9297-781df9402a85 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:46.068638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd338b-d740-4bc1-9297-781df9402a85 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:46.068820 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd338b-d740-4bc1-9297-781df9402a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:46.086972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a0fd338b-d740-4bc1-9297-781df9402a85 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:46.088164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a0fd338b-d740-4bc1-9297-781df9402a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:46.151644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 31/59] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:46.423061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae6188-2f09-4abe-8b79-8a294df564af None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:46.441798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:46.521088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae6188-2f09-4abe-8b79-8a294df564af None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:46.553164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf0a24-925c-4011-a510-4c0d3335b53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:46.553800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf0a24-925c-4011-a510-4c0d3335b53c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:46.554082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf0a24-925c-4011-a510-4c0d3335b53c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:46.554378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf0a24-925c-4011-a510-4c0d3335b53c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:46.555482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf0a24-925c-4011-a510-4c0d3335b53c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:46.556935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf0a24-925c-4011-a510-4c0d3335b53c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:46.558472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 29/60] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:31:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:46.563820 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd338b-d740-4bc1-9297-781df9402a85 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:46.597294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:46.660180 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd338b-d740-4bc1-9297-781df9402a85 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:46.692651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7973c19c-cd07-41c0-ab6c-fb642a44d262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:46.693199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973c19c-cd07-41c0-ab6c-fb642a44d262 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:46.693482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973c19c-cd07-41c0-ab6c-fb642a44d262 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:46.693787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973c19c-cd07-41c0-ab6c-fb642a44d262 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:46.694434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973c19c-cd07-41c0-ab6c-fb642a44d262 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:46.695983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7973c19c-cd07-41c0-ab6c-fb642a44d262 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:46.721589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 32/61] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:31:46 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:31:49.966748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3fa8c-ac51-4c50-8731-c8e614ed2719 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:49.967733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3fa8c-ac51-4c50-8731-c8e614ed2719 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:49.968221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3fa8c-ac51-4c50-8731-c8e614ed2719 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:49.969835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 30/62] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:49.975346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8854364-aea6-4750-8aa8-8e56f2b30ade None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:49.976148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8854364-aea6-4750-8aa8-8e56f2b30ade None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:49.976699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8854364-aea6-4750-8aa8-8e56f2b30ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:49.997007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f8854364-aea6-4750-8aa8-8e56f2b30ade None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:49.998622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f8854364-aea6-4750-8aa8-8e56f2b30ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:50.080661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 33/63] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:50.168087 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a6964-a85c-468a-9e64-1e98fde8f2c9 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:50.168726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a6964-a85c-468a-9e64-1e98fde8f2c9 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:50.169202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a6964-a85c-468a-9e64-1e98fde8f2c9 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:50.170920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 31/64] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:50.176993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a182c-9679-433d-a704-978fe1fc0ee3 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:50.177604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a182c-9679-433d-a704-978fe1fc0ee3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:50.178115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a182c-9679-433d-a704-978fe1fc0ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:50.207749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-426a182c-9679-433d-a704-978fe1fc0ee3 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:50.209282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-426a182c-9679-433d-a704-978fe1fc0ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:50.234408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:50.294443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670d38a-e455-4bc2-b484-e5887e32bb73 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:50.294725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670d38a-e455-4bc2-b484-e5887e32bb73 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:50.295160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670d38a-e455-4bc2-b484-e5887e32bb73 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:50.296641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 32/65] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:50.311279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 34/66] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:50.313577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e191346-67da-4a62-9005-929aa896ee7d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:50.314151 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e191346-67da-4a62-9005-929aa896ee7d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:50.314664 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e191346-67da-4a62-9005-929aa896ee7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:50.346664 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-5e191346-67da-4a62-9005-929aa896ee7d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:50.348210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-5e191346-67da-4a62-9005-929aa896ee7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:50.440166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 33/67] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:50.545949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a182c-9679-433d-a704-978fe1fc0ee3 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:50.601862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633f11e-3e3c-42b4-a307-8414766b40f0 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:50.602500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633f11e-3e3c-42b4-a307-8414766b40f0 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:50.603091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633f11e-3e3c-42b4-a307-8414766b40f0 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:50.604894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 34/68] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:50.610093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcb9fb-d48f-4d79-ab72-f5688a38f581 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:50.610603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcb9fb-d48f-4d79-ab72-f5688a38f581 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:50.611071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcb9fb-d48f-4d79-ab72-f5688a38f581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:50.636309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-7bbcb9fb-d48f-4d79-ab72-f5688a38f581 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:50.637770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7bbcb9fb-d48f-4d79-ab72-f5688a38f581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:50.649335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:50.673108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a182c-9679-433d-a704-978fe1fc0ee3 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:50.713163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 35/69] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:31:50.713361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2eada0-3a54-4c65-b827-3893242d32db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:50.713915 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2eada0-3a54-4c65-b827-3893242d32db None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:50.714208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2eada0-3a54-4c65-b827-3893242d32db None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:50.714546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2eada0-3a54-4c65-b827-3893242d32db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:50.714964 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c360385-b720-4348-93b6-62060af65bd8 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:50.715251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c360385-b720-4348-93b6-62060af65bd8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:50.715523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c360385-b720-4348-93b6-62060af65bd8 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:50.716750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 36/70] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:50.717439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c360385-b720-4348-93b6-62060af65bd8 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:50.733203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2eada0-3a54-4c65-b827-3893242d32db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-4a2eada0-3a54-4c65-b827-3893242d32db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:50.741281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:50.742244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:50.742988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:50.742988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:50.742988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:50.742988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:50.742988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:50.742988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:50.742988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:50.742988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:50.742988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:50.742988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:50.742988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:31:50.742988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:31:50.744956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 35/71] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:31:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:31:50.747149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae6c12-cbe4-4252-aedc-f07f5af6d184 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:50.747644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae6c12-cbe4-4252-aedc-f07f5af6d184 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:50.748379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae6c12-cbe4-4252-aedc-f07f5af6d184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:50.769223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-68ae6c12-cbe4-4252-aedc-f07f5af6d184 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:50.770477 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-68ae6c12-cbe4-4252-aedc-f07f5af6d184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:50.780245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:50.849781 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 36/72] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:50.849940 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c360385-b720-4348-93b6-62060af65bd8 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:50.854643 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae6c12-cbe4-4252-aedc-f07f5af6d184 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:50.890248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7bc43-863d-4ab7-866e-5687c6ffc0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:50.890882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7bc43-863d-4ab7-866e-5687c6ffc0e9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:50.891168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7bc43-863d-4ab7-866e-5687c6ffc0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:50.891444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7bc43-863d-4ab7-866e-5687c6ffc0e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:50.903603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7bc43-863d-4ab7-866e-5687c6ffc0e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-77a7bc43-863d-4ab7-866e-5687c6ffc0e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:50.908209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:50.909039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:31:50.909824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 37/73] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:31:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:31:50.910470 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7bc43-863d-4ab7-866e-5687c6ffc0e9 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:50.924938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:50.981353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae6c12-cbe4-4252-aedc-f07f5af6d184 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:51.021651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be1479-1613-4f16-ac06-d7cd5cdecec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:51.022691 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be1479-1613-4f16-ac06-d7cd5cdecec0 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:51.023780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be1479-1613-4f16-ac06-d7cd5cdecec0 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:51.024254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be1479-1613-4f16-ac06-d7cd5cdecec0 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:51.025051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be1479-1613-4f16-ac06-d7cd5cdecec0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:51.025777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be1479-1613-4f16-ac06-d7cd5cdecec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:51.028009 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be1479-1613-4f16-ac06-d7cd5cdecec0 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:51.029333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7bc43-863d-4ab7-866e-5687c6ffc0e9 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:51.037049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 37/74] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:31:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:51.038014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be1479-1613-4f16-ac06-d7cd5cdecec0 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:51.043222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:51.071471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef557ccf-c6cd-4856-a25c-71519eabd092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:51.072251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef557ccf-c6cd-4856-a25c-71519eabd092 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:51.072379 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef557ccf-c6cd-4856-a25c-71519eabd092 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:51.072651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef557ccf-c6cd-4856-a25c-71519eabd092 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:51.084651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef557ccf-c6cd-4856-a25c-71519eabd092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-ef557ccf-c6cd-4856-a25c-71519eabd092 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:51.089517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:51.090364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:31:51.091109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 38/75] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:31:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:31:51.091745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef557ccf-c6cd-4856-a25c-71519eabd092 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:51.152390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be1479-1613-4f16-ac06-d7cd5cdecec0 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:51.161985 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:51.185168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8539ad0-d7ad-48c2-9bae-30186f8d1220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:51.185919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8539ad0-d7ad-48c2-9bae-30186f8d1220 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:51.186204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8539ad0-d7ad-48c2-9bae-30186f8d1220 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:51.186473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8539ad0-d7ad-48c2-9bae-30186f8d1220 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:51.187124 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8539ad0-d7ad-48c2-9bae-30186f8d1220 None admin] RBAC: Adding query filter params (name=member) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:51.187425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8539ad0-d7ad-48c2-9bae-30186f8d1220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:51.188808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8539ad0-d7ad-48c2-9bae-30186f8d1220 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:51.196873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 38/76] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:31:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:51.197373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef557ccf-c6cd-4856-a25c-71519eabd092 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:51.197651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8539ad0-d7ad-48c2-9bae-30186f8d1220 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:51.233466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a097b-65a4-4440-8a70-11e3dd713a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:51.234125 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a097b-65a4-4440-8a70-11e3dd713a17 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:51.234406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a097b-65a4-4440-8a70-11e3dd713a17 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:51.234676 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a097b-65a4-4440-8a70-11e3dd713a17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:51.235797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a097b-65a4-4440-8a70-11e3dd713a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:51.237252 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a097b-65a4-4440-8a70-11e3dd713a17 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:51.238840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 39/77] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:31:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:51.241786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a097b-65a4-4440-8a70-11e3dd713a17 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:51.263065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:51.298140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8539ad0-d7ad-48c2-9bae-30186f8d1220 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:51.330352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ce73a-c4d2-4ac2-84f8-26f52b6e0c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:51.330981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ce73a-c4d2-4ac2-84f8-26f52b6e0c4d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:51.331264 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ce73a-c4d2-4ac2-84f8-26f52b6e0c4d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:51.331538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ce73a-c4d2-4ac2-84f8-26f52b6e0c4d None admin] PATH_INFO: `/v3/users/f84b5144054d4e158a838b4ae551996f` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:51.332685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ce73a-c4d2-4ac2-84f8-26f52b6e0c4d None admin] RBAC: Authorizing `identity:get_user(user_id=f84b5144054d4e158a838b4ae551996f)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:51.334166 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ce73a-c4d2-4ac2-84f8-26f52b6e0c4d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:51.336708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 39/78] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:31:51 2022] GET /identity/v3/users/f84b5144054d4e158a838b4ae551996f => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:51.336980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ce73a-c4d2-4ac2-84f8-26f52b6e0c4d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:51.339367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a097b-65a4-4440-8a70-11e3dd713a17 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:51.372185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5524ee-9aef-4380-ac42-3da4f067380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:51.372785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5524ee-9aef-4380-ac42-3da4f067380c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:51.373065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5524ee-9aef-4380-ac42-3da4f067380c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:51.373335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5524ee-9aef-4380-ac42-3da4f067380c None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:51.374025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5524ee-9aef-4380-ac42-3da4f067380c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:51.374328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5524ee-9aef-4380-ac42-3da4f067380c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:51.375694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5524ee-9aef-4380-ac42-3da4f067380c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:51.377046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:51.384480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 40/79] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:31:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:51.385263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5524ee-9aef-4380-ac42-3da4f067380c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:51.437963 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ce73a-c4d2-4ac2-84f8-26f52b6e0c4d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:51.470464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0b1b2-c1d1-4028-927a-210da3b68960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:51.471119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0b1b2-c1d1-4028-927a-210da3b68960 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:51.471405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0b1b2-c1d1-4028-927a-210da3b68960 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:51.471682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0b1b2-c1d1-4028-927a-210da3b68960 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:51.473465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:51.482394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5524ee-9aef-4380-ac42-3da4f067380c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:51.484016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0b1b2-c1d1-4028-927a-210da3b68960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-f4b0b1b2-c1d1-4028-927a-210da3b68960 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:51.489246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:51.490289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:31:51.491069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 40/80] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:31:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:31:51.494452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0b1b2-c1d1-4028-927a-210da3b68960 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:51.515544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a0dd4-ae8a-4865-ac3f-03afd186d930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:51.516176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a0dd4-ae8a-4865-ac3f-03afd186d930 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:51.516463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a0dd4-ae8a-4865-ac3f-03afd186d930 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:51.516742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a0dd4-ae8a-4865-ac3f-03afd186d930 None admin] PATH_INFO: `/v3/users/902e1680a1bd4607ad018968c6bfe87c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:51.530139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a0dd4-ae8a-4865-ac3f-03afd186d930 None admin] RBAC: Authorizing `identity:get_user(user_id=902e1680a1bd4607ad018968c6bfe87c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:51.531642 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a0dd4-ae8a-4865-ac3f-03afd186d930 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:51.533284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 41/81] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:31:51 2022] GET /identity/v3/users/902e1680a1bd4607ad018968c6bfe87c => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:51.534088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a0dd4-ae8a-4865-ac3f-03afd186d930 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:51.588247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:51.596148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0b1b2-c1d1-4028-927a-210da3b68960 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:51.628214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623d3c7-7565-471f-bee4-3d206b0f0465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:51.628815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623d3c7-7565-471f-bee4-3d206b0f0465 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:51.629093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623d3c7-7565-471f-bee4-3d206b0f0465 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:51.629369 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623d3c7-7565-471f-bee4-3d206b0f0465 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:51.630049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623d3c7-7565-471f-bee4-3d206b0f0465 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:51.631478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623d3c7-7565-471f-bee4-3d206b0f0465 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:51.632000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a0dd4-ae8a-4865-ac3f-03afd186d930 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:51.652706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 41/82] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:31:51 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:31:51.653538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623d3c7-7565-471f-bee4-3d206b0f0465 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:51.689750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c0041-7b7c-4d87-97dc-92772b7b5833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:51.690294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c0041-7b7c-4d87-97dc-92772b7b5833 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:51.690575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c0041-7b7c-4d87-97dc-92772b7b5833 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:51.690903 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c0041-7b7c-4d87-97dc-92772b7b5833 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:51.714179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:51.771554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c0041-7b7c-4d87-97dc-92772b7b5833 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:51.773081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c0041-7b7c-4d87-97dc-92772b7b5833 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:51.775417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623d3c7-7565-471f-bee4-3d206b0f0465 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:51.813465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06a9d2-b353-4588-805d-5f3345d8e407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:51.814449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06a9d2-b353-4588-805d-5f3345d8e407 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:51.814909 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06a9d2-b353-4588-805d-5f3345d8e407 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:51.815384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06a9d2-b353-4588-805d-5f3345d8e407 None admin] PATH_INFO: `/v3/users/902e1680a1bd4607ad018968c6bfe87c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:51.816731 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06a9d2-b353-4588-805d-5f3345d8e407 None admin] RBAC: Authorizing `identity:get_user(user_id=902e1680a1bd4607ad018968c6bfe87c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:51.818424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06a9d2-b353-4588-805d-5f3345d8e407 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:51.820317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 42/83] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:31:51 2022] GET /identity/v3/users/902e1680a1bd4607ad018968c6bfe87c => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:51.821247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06a9d2-b353-4588-805d-5f3345d8e407 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:51.838864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:51.899161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 42/84] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:31:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:31:51.903539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c0041-7b7c-4d87-97dc-92772b7b5833 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:51.950216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06a9d2-b353-4588-805d-5f3345d8e407 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:51.971562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:51.996446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfff88b-c271-4a21-a7b6-e29d29bdd222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:51.996446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfff88b-c271-4a21-a7b6-e29d29bdd222 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:51.996446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfff88b-c271-4a21-a7b6-e29d29bdd222 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:51.996446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfff88b-c271-4a21-a7b6-e29d29bdd222 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:51.997232 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfff88b-c271-4a21-a7b6-e29d29bdd222 None admin] RBAC: Adding query filter params (name=member) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:51.997232 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfff88b-c271-4a21-a7b6-e29d29bdd222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:51.998300 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfff88b-c271-4a21-a7b6-e29d29bdd222 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:52.009803 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 43/85] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:31:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:52.010692 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfff88b-c271-4a21-a7b6-e29d29bdd222 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:52.030635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c0041-7b7c-4d87-97dc-92772b7b5833 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:52.076162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ed5b1-bc77-48a4-ad8f-77d494b47660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:52.077328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ed5b1-bc77-48a4-ad8f-77d494b47660 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:52.077615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ed5b1-bc77-48a4-ad8f-77d494b47660 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:52.077923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ed5b1-bc77-48a4-ad8f-77d494b47660 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:52.110702 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:52.142977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfff88b-c271-4a21-a7b6-e29d29bdd222 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:52.169038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ed5b1-bc77-48a4-ad8f-77d494b47660 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:52.171417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ed5b1-bc77-48a4-ad8f-77d494b47660 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:52.189105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62c3bf-24ef-4dbe-98a6-b8169fe646c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:52.190348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62c3bf-24ef-4dbe-98a6-b8169fe646c3 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:52.191314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62c3bf-24ef-4dbe-98a6-b8169fe646c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:52.192105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62c3bf-24ef-4dbe-98a6-b8169fe646c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:52.259768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:52.279976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62c3bf-24ef-4dbe-98a6-b8169fe646c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:52.281506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62c3bf-24ef-4dbe-98a6-b8169fe646c3 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:52.300641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 43/86] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:31:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:31:52.301959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ed5b1-bc77-48a4-ad8f-77d494b47660 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:52.359057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:52.392706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 44/87] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:31:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:31:52.393622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62c3bf-24ef-4dbe-98a6-b8169fe646c3 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:52.408338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ed5b1-bc77-48a4-ad8f-77d494b47660 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:52.446036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054be6e-92a7-4ab0-86ec-837415485667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:52.447090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054be6e-92a7-4ab0-86ec-837415485667 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:52.447568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054be6e-92a7-4ab0-86ec-837415485667 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:52.448036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054be6e-92a7-4ab0-86ec-837415485667 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:52.449465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a054be6e-92a7-4ab0-86ec-837415485667 None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:52.451165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a054be6e-92a7-4ab0-86ec-837415485667 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:52.453240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 44/88] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:31:52 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:52.454322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054be6e-92a7-4ab0-86ec-837415485667 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:52.479823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:52.501067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62c3bf-24ef-4dbe-98a6-b8169fe646c3 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:52.538102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad911b-7e3c-42e0-b1eb-dfe793b8f969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:52.539261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad911b-7e3c-42e0-b1eb-dfe793b8f969 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:52.539559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad911b-7e3c-42e0-b1eb-dfe793b8f969 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:52.539832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad911b-7e3c-42e0-b1eb-dfe793b8f969 None admin] PATH_INFO: `/v3/users/902e1680a1bd4607ad018968c6bfe87c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:52.541343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad911b-7e3c-42e0-b1eb-dfe793b8f969 None admin] RBAC: Authorizing `identity:get_user(user_id=902e1680a1bd4607ad018968c6bfe87c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:52.542651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad911b-7e3c-42e0-b1eb-dfe793b8f969 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:52.544577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 45/89] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:31:52 2022] GET /identity/v3/users/902e1680a1bd4607ad018968c6bfe87c => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:52.545186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad911b-7e3c-42e0-b1eb-dfe793b8f969 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:52.569778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054be6e-92a7-4ab0-86ec-837415485667 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:52.594326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:52.608186 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d25e6-cecc-4b43-af95-7d006d829537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:52.609202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d25e6-cecc-4b43-af95-7d006d829537 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:52.609734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d25e6-cecc-4b43-af95-7d006d829537 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:52.610225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d25e6-cecc-4b43-af95-7d006d829537 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:52.611749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d25e6-cecc-4b43-af95-7d006d829537 None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:52.613409 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d25e6-cecc-4b43-af95-7d006d829537 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:52.615823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 45/90] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:31:52 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:52.617012 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d25e6-cecc-4b43-af95-7d006d829537 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:52.656663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad911b-7e3c-42e0-b1eb-dfe793b8f969 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:52.695648 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b780a-1fbb-4832-821f-7f8b9998caa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:52.696403 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b780a-1fbb-4832-821f-7f8b9998caa2 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:52.696602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b780a-1fbb-4832-821f-7f8b9998caa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:52.696900 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b780a-1fbb-4832-821f-7f8b9998caa2 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:52.698147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b780a-1fbb-4832-821f-7f8b9998caa2 None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:52.700316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b780a-1fbb-4832-821f-7f8b9998caa2 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:52.701573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 46/91] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:31:52 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:52.702363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b780a-1fbb-4832-821f-7f8b9998caa2 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:52.721645 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:52.735710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d25e6-cecc-4b43-af95-7d006d829537 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:52.774411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea790e3-a1ba-4482-bdae-a6572fb5ad22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:52.775370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea790e3-a1ba-4482-bdae-a6572fb5ad22 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:52.775975 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea790e3-a1ba-4482-bdae-a6572fb5ad22 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:52.776443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea790e3-a1ba-4482-bdae-a6572fb5ad22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:52.777465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea790e3-a1ba-4482-bdae-a6572fb5ad22 None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=1244700aec90459fbdbde9822e93a110, user.id=f84b5144054d4e158a838b4ae551996f) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:52.777999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea790e3-a1ba-4482-bdae-a6572fb5ad22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:52.779998 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea790e3-a1ba-4482-bdae-a6572fb5ad22 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:52.786446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 46/92] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:31:52 2022] GET /identity/v3/role_assignments?user.id=f84b5144054d4e158a838b4ae551996f&scope.project.id=1244700aec90459fbdbde9822e93a110&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:52.787630 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea790e3-a1ba-4482-bdae-a6572fb5ad22 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:52.814145 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b780a-1fbb-4832-821f-7f8b9998caa2 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:52.850568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:52.856866 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1c145-efae-49db-9845-839266710d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:52.857513 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1c145-efae-49db-9845-839266710d16 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:52.857817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1c145-efae-49db-9845-839266710d16 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:52.858158 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1c145-efae-49db-9845-839266710d16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:52.904093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea790e3-a1ba-4482-bdae-a6572fb5ad22 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:52.937072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1c145-efae-49db-9845-839266710d16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:52.938560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1c145-efae-49db-9845-839266710d16 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:52.942762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02a45c-0687-4a72-a8b8-0834c8a46982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:52.943599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02a45c-0687-4a72-a8b8-0834c8a46982 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:52.944080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02a45c-0687-4a72-a8b8-0834c8a46982 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:52.944563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a02a45c-0687-4a72-a8b8-0834c8a46982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:52.946123 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02a45c-0687-4a72-a8b8-0834c8a46982 None admin] RBAC: Adding query filter params (role.id=efddf5f5222644b18b6836117621adf8, scope.project.id=1244700aec90459fbdbde9822e93a110, user.id=902e1680a1bd4607ad018968c6bfe87c) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:52.946643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02a45c-0687-4a72-a8b8-0834c8a46982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:52.948435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a02a45c-0687-4a72-a8b8-0834c8a46982 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:52.955527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 47/93] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:31:52 2022] GET /identity/v3/role_assignments?user.id=902e1680a1bd4607ad018968c6bfe87c&scope.project.id=1244700aec90459fbdbde9822e93a110&role.id=efddf5f5222644b18b6836117621adf8 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:52.956837 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02a45c-0687-4a72-a8b8-0834c8a46982 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:52.957896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:53.056571 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 47/94] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:31:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:31:53.062643 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1c145-efae-49db-9845-839266710d16 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:53.081741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a02a45c-0687-4a72-a8b8-0834c8a46982 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:53.122018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c79f74-4099-44a3-98f5-fae7f26dba5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:53.122906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c79f74-4099-44a3-98f5-fae7f26dba5b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:53.123395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c79f74-4099-44a3-98f5-fae7f26dba5b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:53.123865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c79f74-4099-44a3-98f5-fae7f26dba5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:53.124793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c79f74-4099-44a3-98f5-fae7f26dba5b None admin] RBAC: Adding query filter params (role.id=c0bf52b447564f769d707e96facb20a1, scope.project.id=1244700aec90459fbdbde9822e93a110, user.id=902e1680a1bd4607ad018968c6bfe87c) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:53.125301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c79f74-4099-44a3-98f5-fae7f26dba5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:53.127024 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c79f74-4099-44a3-98f5-fae7f26dba5b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:53.133292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 48/95] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:31:52 2022] GET /identity/v3/role_assignments?user.id=902e1680a1bd4607ad018968c6bfe87c&scope.project.id=1244700aec90459fbdbde9822e93a110&role.id=c0bf52b447564f769d707e96facb20a1 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:53.133639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:53.206607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1c145-efae-49db-9845-839266710d16 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:53.237001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:53.266625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800aba7-77fd-4378-9b3f-f8f708e1b1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:53.267291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800aba7-77fd-4378-9b3f-f8f708e1b1dd None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:53.267578 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800aba7-77fd-4378-9b3f-f8f708e1b1dd None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:53.267853 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800aba7-77fd-4378-9b3f-f8f708e1b1dd None admin] PATH_INFO: `/v3/projects/050f7c5b157546cc884011378cf96ce1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:53.270301 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800aba7-77fd-4378-9b3f-f8f708e1b1dd None admin] RBAC: Authorizing `identity:get_project(project_id=050f7c5b157546cc884011378cf96ce1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:53.271815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800aba7-77fd-4378-9b3f-f8f708e1b1dd None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:53.274192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 48/96] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:31:53 2022] GET /identity/v3/projects/050f7c5b157546cc884011378cf96ce1 => generated 312 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:53.280655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c79f74-4099-44a3-98f5-fae7f26dba5b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:53.423522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c79f74-4099-44a3-98f5-fae7f26dba5b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:53.462518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23a8a7-8ab6-453e-bf75-6ace1e26488a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:53.463489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23a8a7-8ab6-453e-bf75-6ace1e26488a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:53.464037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23a8a7-8ab6-453e-bf75-6ace1e26488a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:53.464508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23a8a7-8ab6-453e-bf75-6ace1e26488a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:53.465437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23a8a7-8ab6-453e-bf75-6ace1e26488a None admin] RBAC: Adding query filter params (role.id=efddf5f5222644b18b6836117621adf8, scope.project.id=050f7c5b157546cc884011378cf96ce1, user.id=902e1680a1bd4607ad018968c6bfe87c) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:53.466041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23a8a7-8ab6-453e-bf75-6ace1e26488a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:53.467802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23a8a7-8ab6-453e-bf75-6ace1e26488a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:53.474258 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 49/97] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:31:53 2022] GET /identity/v3/role_assignments?user.id=902e1680a1bd4607ad018968c6bfe87c&scope.project.id=050f7c5b157546cc884011378cf96ce1&role.id=efddf5f5222644b18b6836117621adf8 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:54.278006 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c42302-bc13-48f9-bedb-a78a12a018b8 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:54.278520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c42302-bc13-48f9-bedb-a78a12a018b8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:54.278594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c42302-bc13-48f9-bedb-a78a12a018b8 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:54.279896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 49/98] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:54.285132 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da509d18-6204-4629-b40f-c4e6b7ba6e77 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:54.285548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da509d18-6204-4629-b40f-c4e6b7ba6e77 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:54.285750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da509d18-6204-4629-b40f-c4e6b7ba6e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:54.307000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-da509d18-6204-4629-b40f-c4e6b7ba6e77 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:54.308227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-da509d18-6204-4629-b40f-c4e6b7ba6e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:54.363132 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:54.392748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 50/99] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:54.474424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94c894-58b7-4787-a412-c8aade6a2b1e None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:54.474872 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94c894-58b7-4787-a412-c8aade6a2b1e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:54.475253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94c894-58b7-4787-a412-c8aade6a2b1e None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:54.477426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 50/100] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:54.483138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191a0c5-9cbf-4ff6-9ae2-22f11436035f None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:54.483429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191a0c5-9cbf-4ff6-9ae2-22f11436035f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:54.483710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c191a0c5-9cbf-4ff6-9ae2-22f11436035f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:54.505621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-c191a0c5-9cbf-4ff6-9ae2-22f11436035f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:54.506839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c191a0c5-9cbf-4ff6-9ae2-22f11436035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:54.589242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 51/101] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:54.775665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94c894-58b7-4787-a412-c8aade6a2b1e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:54.895925 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94c894-58b7-4787-a412-c8aade6a2b1e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:54.933310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ac75b-539d-4fd7-9d5b-e3ca68432294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:54.933973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ac75b-539d-4fd7-9d5b-e3ca68432294 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:54.934254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ac75b-539d-4fd7-9d5b-e3ca68432294 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:54.934531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ac75b-539d-4fd7-9d5b-e3ca68432294 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:54.945747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ac75b-539d-4fd7-9d5b-e3ca68432294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-e08ac75b-539d-4fd7-9d5b-e3ca68432294 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:54.950518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:54.951433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:31:54.952163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 51/102] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:31:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:31:54.958706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191a0c5-9cbf-4ff6-9ae2-22f11436035f None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:55.080165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ac75b-539d-4fd7-9d5b-e3ca68432294 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:55.104738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c191a0c5-9cbf-4ff6-9ae2-22f11436035f None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:55.115563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:55.157433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e75eb-d2e5-44e5-b2e6-6bed6430afbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:55.158100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e75eb-d2e5-44e5-b2e6-6bed6430afbf None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:55.158389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e75eb-d2e5-44e5-b2e6-6bed6430afbf None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:55.158669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e75eb-d2e5-44e5-b2e6-6bed6430afbf None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:55.159414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e75eb-d2e5-44e5-b2e6-6bed6430afbf None admin] RBAC: Adding query filter params (name=reader) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:55.159721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e75eb-d2e5-44e5-b2e6-6bed6430afbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:55.161229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e75eb-d2e5-44e5-b2e6-6bed6430afbf None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:55.176001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 52/103] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:31:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:55.181076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:55.182327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e75eb-d2e5-44e5-b2e6-6bed6430afbf None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:55.327833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ac75b-539d-4fd7-9d5b-e3ca68432294 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:55.349547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e75eb-d2e5-44e5-b2e6-6bed6430afbf None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:55.412066 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d1520-30ea-4a03-b7d3-d4c2e6900b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:55.412627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1520-30ea-4a03-b7d3-d4c2e6900b8e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:55.412915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1520-30ea-4a03-b7d3-d4c2e6900b8e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:55.413196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1520-30ea-4a03-b7d3-d4c2e6900b8e None admin] PATH_INFO: `/v3/users/258ae95275494c5c8bb8e98d920d2c97` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:55.420672 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2357777-b1b4-4c77-98e3-5be67a5a724f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:55.421313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2357777-b1b4-4c77-98e3-5be67a5a724f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:55.421594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2357777-b1b4-4c77-98e3-5be67a5a724f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:55.421894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2357777-b1b4-4c77-98e3-5be67a5a724f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:55.423017 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2357777-b1b4-4c77-98e3-5be67a5a724f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:55.424465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2357777-b1b4-4c77-98e3-5be67a5a724f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:55.426229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 52/104] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:31:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:55.432830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d1520-30ea-4a03-b7d3-d4c2e6900b8e None admin] RBAC: Authorizing `identity:get_user(user_id=258ae95275494c5c8bb8e98d920d2c97)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:55.432964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2357777-b1b4-4c77-98e3-5be67a5a724f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:55.434689 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d1520-30ea-4a03-b7d3-d4c2e6900b8e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:55.436399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 53/105] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:31:55 2022] GET /identity/v3/users/258ae95275494c5c8bb8e98d920d2c97 => generated 289 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:55.441938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d1520-30ea-4a03-b7d3-d4c2e6900b8e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:55.483591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:55.630747 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d1520-30ea-4a03-b7d3-d4c2e6900b8e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:55.678765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2357777-b1b4-4c77-98e3-5be67a5a724f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:55.710054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:55.712791 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64012555-9034-410c-86b0-dd5d3d52e193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:55.713418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64012555-9034-410c-86b0-dd5d3d52e193 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:55.713720 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64012555-9034-410c-86b0-dd5d3d52e193 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:55.713992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64012555-9034-410c-86b0-dd5d3d52e193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:55.743863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370910b9-3905-4063-8949-aace21a7f984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:55.744548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370910b9-3905-4063-8949-aace21a7f984 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:55.744837 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370910b9-3905-4063-8949-aace21a7f984 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:55.745115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370910b9-3905-4063-8949-aace21a7f984 None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:55.745811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370910b9-3905-4063-8949-aace21a7f984 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:55.747352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370910b9-3905-4063-8949-aace21a7f984 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:55.793639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 53/106] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:31:55 2022] POST /identity/v3/projects => generated 302 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:31:55.850063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64012555-9034-410c-86b0-dd5d3d52e193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:55.852192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64012555-9034-410c-86b0-dd5d3d52e193 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:55.880646 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:55.958203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 54/107] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:31:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:31:55.964275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370910b9-3905-4063-8949-aace21a7f984 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:56.071812 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370910b9-3905-4063-8949-aace21a7f984 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:56.107884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eeced5-14a9-40cf-9c95-cd14565ab63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:56.108530 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeced5-14a9-40cf-9c95-cd14565ab63b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:56.108813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeced5-14a9-40cf-9c95-cd14565ab63b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:56.109096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeced5-14a9-40cf-9c95-cd14565ab63b None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:56.110320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eeced5-14a9-40cf-9c95-cd14565ab63b None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:56.111781 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eeced5-14a9-40cf-9c95-cd14565ab63b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:56.113506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 54/108] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:31:55 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:56.120277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64012555-9034-410c-86b0-dd5d3d52e193 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:56.229625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64012555-9034-410c-86b0-dd5d3d52e193 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:56.256793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:56.267306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a08f7-a9e7-4ebc-a198-ca8ba0a770b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:56.268814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a08f7-a9e7-4ebc-a198-ca8ba0a770b0 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:56.269672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a08f7-a9e7-4ebc-a198-ca8ba0a770b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:56.274570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a08f7-a9e7-4ebc-a198-ca8ba0a770b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:56.276041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a08f7-a9e7-4ebc-a198-ca8ba0a770b0 None admin] RBAC: Adding query filter params (role.id=015747495a144cb0976ad270d651b48f, scope.project.id=1244700aec90459fbdbde9822e93a110, user.id=258ae95275494c5c8bb8e98d920d2c97) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:56.276622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a08f7-a9e7-4ebc-a198-ca8ba0a770b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:56.279802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a08f7-a9e7-4ebc-a198-ca8ba0a770b0 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:56.286624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 55/109] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:31:56 2022] GET /identity/v3/role_assignments?user.id=258ae95275494c5c8bb8e98d920d2c97&scope.project.id=1244700aec90459fbdbde9822e93a110&role.id=015747495a144cb0976ad270d651b48f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:56.302937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22130546-38aa-401f-a284-b5a0a35e5395 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:56.303200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22130546-38aa-401f-a284-b5a0a35e5395 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:56.303475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22130546-38aa-401f-a284-b5a0a35e5395 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:56.304930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 55/110] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:56.310786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7c398-eae6-452b-b794-35a2bb87eca8 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:56.311138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7c398-eae6-452b-b794-35a2bb87eca8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:56.311394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7c398-eae6-452b-b794-35a2bb87eca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:56.331144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-f9c7c398-eae6-452b-b794-35a2bb87eca8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:56.332373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f9c7c398-eae6-452b-b794-35a2bb87eca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:56.410968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 56/111] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:56.694293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dec01-036b-464c-8ba8-ed1ea22bb214 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:56.694810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dec01-036b-464c-8ba8-ed1ea22bb214 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:56.694884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dec01-036b-464c-8ba8-ed1ea22bb214 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:56.697259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 56/112] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:56.703641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233dec01-036b-464c-8ba8-ed1ea22bb214 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:56.706537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699db095-d202-4c0f-8944-55e131cee661 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:56.706849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699db095-d202-4c0f-8944-55e131cee661 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:56.707129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699db095-d202-4c0f-8944-55e131cee661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:56.742454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-699db095-d202-4c0f-8944-55e131cee661 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:56.743659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-699db095-d202-4c0f-8944-55e131cee661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:56.761740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:56.850124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 57/113] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:56.852197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f62dde-8601-44f4-81a4-6e03e92d3250 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:56.852664 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f62dde-8601-44f4-81a4-6e03e92d3250 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:56.853107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f62dde-8601-44f4-81a4-6e03e92d3250 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:56.854791 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 58/114] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:56.856147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f3ce3-571a-471b-a4e8-c8aecce6fa30 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:56.856735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f3ce3-571a-471b-a4e8-c8aecce6fa30 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:56.857219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f3ce3-571a-471b-a4e8-c8aecce6fa30 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:56.858700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 59/115] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:56.864302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6384994-4d37-4ea5-b829-b95b7343e8b8 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:56.864822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6384994-4d37-4ea5-b829-b95b7343e8b8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:56.865411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6384994-4d37-4ea5-b829-b95b7343e8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:56.882396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233dec01-036b-464c-8ba8-ed1ea22bb214 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:56.886292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b6384994-4d37-4ea5-b829-b95b7343e8b8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:56.887742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b6384994-4d37-4ea5-b829-b95b7343e8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:56.942397 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:56.961481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09819164-65a4-4301-8b31-b92333711104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:56.963332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09819164-65a4-4301-8b31-b92333711104 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:56.963932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09819164-65a4-4301-8b31-b92333711104 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:56.964917 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09819164-65a4-4301-8b31-b92333711104 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:56.980765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 60/116] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:56.982316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09819164-65a4-4301-8b31-b92333711104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:56.982590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e178ef4-426c-4891-89ad-bcbd8ef30e51 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:56.983059 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e178ef4-426c-4891-89ad-bcbd8ef30e51 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:56.983540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e178ef4-426c-4891-89ad-bcbd8ef30e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-09819164-65a4-4301-8b31-b92333711104 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:57.001986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:57.002899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:31:57.003642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 57/117] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:31:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:31:57.010451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-1e178ef4-426c-4891-89ad-bcbd8ef30e51 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:57.011892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1e178ef4-426c-4891-89ad-bcbd8ef30e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:57.017771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09819164-65a4-4301-8b31-b92333711104 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:57.144056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 61/118] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:31:57.160918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e178ef4-426c-4891-89ad-bcbd8ef30e51 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:57.214168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:57.226344 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09819164-65a4-4301-8b31-b92333711104 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:57.265563 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecaaff1-3980-4e72-b17f-63ffae5f72ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:57.266212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaaff1-3980-4e72-b17f-63ffae5f72ca None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:57.266653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaaff1-3980-4e72-b17f-63ffae5f72ca None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:57.267215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaaff1-3980-4e72-b17f-63ffae5f72ca None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:57.268023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecaaff1-3980-4e72-b17f-63ffae5f72ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:57.268469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecaaff1-3980-4e72-b17f-63ffae5f72ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:57.270099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecaaff1-3980-4e72-b17f-63ffae5f72ca None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:57.278816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 58/119] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:31:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:57.285094 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecaaff1-3980-4e72-b17f-63ffae5f72ca None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:57.329536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e178ef4-426c-4891-89ad-bcbd8ef30e51 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:57.344020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:57.379319 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e40a7-1925-4857-897f-bb1973f3aeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:57.381046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e40a7-1925-4857-897f-bb1973f3aeac None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:57.381778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e40a7-1925-4857-897f-bb1973f3aeac None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:57.382474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e40a7-1925-4857-897f-bb1973f3aeac None admin] PATH_INFO: `/v3/roles/member` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:57.404128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e40a7-1925-4857-897f-bb1973f3aeac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-b22e40a7-1925-4857-897f-bb1973f3aeac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:57.410219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:57.411548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:57.412346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:57.412346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:57.412346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:57.412346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:57.412346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:57.412346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:57.412346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:57.412346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:57.412346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:57.412346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:57.412346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:31:57.412346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:31:57.416176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 62/120] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:31:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:31:57.418099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e40a7-1925-4857-897f-bb1973f3aeac None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:57.489235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecaaff1-3980-4e72-b17f-63ffae5f72ca None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:57.529522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702ed56-062c-4ce9-92f3-1ec25b78bb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:57.530631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702ed56-062c-4ce9-92f3-1ec25b78bb52 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:57.531115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702ed56-062c-4ce9-92f3-1ec25b78bb52 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:57.531727 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702ed56-062c-4ce9-92f3-1ec25b78bb52 None admin] PATH_INFO: `/v3/users/f84b5144054d4e158a838b4ae551996f` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:57.533238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702ed56-062c-4ce9-92f3-1ec25b78bb52 None admin] RBAC: Authorizing `identity:get_user(user_id=f84b5144054d4e158a838b4ae551996f)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:57.535153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702ed56-062c-4ce9-92f3-1ec25b78bb52 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:57.537561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 59/121] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:31:57 2022] GET /identity/v3/users/f84b5144054d4e158a838b4ae551996f => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:31:57.539417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702ed56-062c-4ce9-92f3-1ec25b78bb52 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:57.565270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e40a7-1925-4857-897f-bb1973f3aeac None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:57.568994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:57.613168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a45eb-2968-4560-95d9-c72b35932bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:57.613841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a45eb-2968-4560-95d9-c72b35932bb7 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:57.614131 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a45eb-2968-4560-95d9-c72b35932bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:57.614557 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a45eb-2968-4560-95d9-c72b35932bb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:57.633884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a45eb-2968-4560-95d9-c72b35932bb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-639a45eb-2968-4560-95d9-c72b35932bb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:57.640465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:57.641292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:31:57.642111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 63/122] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:31:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:31:57.642743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a45eb-2968-4560-95d9-c72b35932bb7 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:57.675647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702ed56-062c-4ce9-92f3-1ec25b78bb52 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:57.690685 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:57.716090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed7011-3116-417f-ab33-0b6b9a5df62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:57.716763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed7011-3116-417f-ab33-0b6b9a5df62f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:57.717052 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed7011-3116-417f-ab33-0b6b9a5df62f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:57.717331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed7011-3116-417f-ab33-0b6b9a5df62f None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:57.718478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed7011-3116-417f-ab33-0b6b9a5df62f None admin] RBAC: Adding query filter params (name=member) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:57.718795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed7011-3116-417f-ab33-0b6b9a5df62f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:57.720512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed7011-3116-417f-ab33-0b6b9a5df62f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:57.735320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 60/123] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:31:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:57.736342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed7011-3116-417f-ab33-0b6b9a5df62f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:57.794616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a45eb-2968-4560-95d9-c72b35932bb7 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:57.830560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c88cfc-3d94-463e-abb2-885926baed6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:57.830984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c88cfc-3d94-463e-abb2-885926baed6a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:57.831331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c88cfc-3d94-463e-abb2-885926baed6a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:57.831616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c88cfc-3d94-463e-abb2-885926baed6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:57.837636 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:57.842614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c88cfc-3d94-463e-abb2-885926baed6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-e2c88cfc-3d94-463e-abb2-885926baed6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:57.847520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:57.848382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:57.849137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:57.849137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:57.849137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:57.849137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:57.849137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:57.849137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:57.849137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:57.849137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:57.849137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:57.849137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:57.849137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:31:57.849137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:31:57.853910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 64/124] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:31:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:31:57.854808 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c88cfc-3d94-463e-abb2-885926baed6a None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:57.861655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed7011-3116-417f-ab33-0b6b9a5df62f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:57.900992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ffb9-a737-4854-8672-a92e3c69da70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:57.902021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ffb9-a737-4854-8672-a92e3c69da70 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:57.902505 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ffb9-a737-4854-8672-a92e3c69da70 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:57.902979 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ffb9-a737-4854-8672-a92e3c69da70 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:57.904339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351ffb9-a737-4854-8672-a92e3c69da70 None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:57.906609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351ffb9-a737-4854-8672-a92e3c69da70 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:57.909050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 61/125] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:31:57 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:57.910116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ffb9-a737-4854-8672-a92e3c69da70 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:57.953062 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:57.964581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c88cfc-3d94-463e-abb2-885926baed6a None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:58.000887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28274e6-f163-4c21-9558-eac8f82c4137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:58.001503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28274e6-f163-4c21-9558-eac8f82c4137 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:58.001915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28274e6-f163-4c21-9558-eac8f82c4137 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:58.002200 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28274e6-f163-4c21-9558-eac8f82c4137 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:58.002880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28274e6-f163-4c21-9558-eac8f82c4137 None admin] RBAC: Adding query filter params (name=member) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:58.003248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28274e6-f163-4c21-9558-eac8f82c4137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:58.004717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28274e6-f163-4c21-9558-eac8f82c4137 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:58.013559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 65/126] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:31:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:58.018226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28274e6-f163-4c21-9558-eac8f82c4137 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:58.036206 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ffb9-a737-4854-8672-a92e3c69da70 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:58.074962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bf055-7960-46b7-a238-d529222b3806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:58.076068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bf055-7960-46b7-a238-d529222b3806 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:58.076528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bf055-7960-46b7-a238-d529222b3806 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:58.076957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bf055-7960-46b7-a238-d529222b3806 None admin] PATH_INFO: `/v3/users/902e1680a1bd4607ad018968c6bfe87c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:58.078292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bf055-7960-46b7-a238-d529222b3806 None admin] RBAC: Authorizing `identity:get_user(user_id=902e1680a1bd4607ad018968c6bfe87c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:58.079884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bf055-7960-46b7-a238-d529222b3806 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:58.082402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 62/127] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:31:57 2022] GET /identity/v3/users/902e1680a1bd4607ad018968c6bfe87c => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:58.083393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bf055-7960-46b7-a238-d529222b3806 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:58.115235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:58.136052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28274e6-f163-4c21-9558-eac8f82c4137 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:58.177824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a82dc-b6e1-4556-bc7a-50062c48a043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:58.178607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a82dc-b6e1-4556-bc7a-50062c48a043 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:58.178988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a82dc-b6e1-4556-bc7a-50062c48a043 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:58.179399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a82dc-b6e1-4556-bc7a-50062c48a043 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:58.180170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a82dc-b6e1-4556-bc7a-50062c48a043 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:31:58.180584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a82dc-b6e1-4556-bc7a-50062c48a043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:58.182344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a82dc-b6e1-4556-bc7a-50062c48a043 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:58.190681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 66/128] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:31:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:58.195854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a82dc-b6e1-4556-bc7a-50062c48a043 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:58.208149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bf055-7960-46b7-a238-d529222b3806 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:58.213535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:58.251350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833bacd-6e11-4cf3-80df-37317adb7761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:58.252259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833bacd-6e11-4cf3-80df-37317adb7761 None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:58.252679 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833bacd-6e11-4cf3-80df-37317adb7761 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:58.253090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833bacd-6e11-4cf3-80df-37317adb7761 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110/users/f84b5144054d4e158a838b4ae551996f/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:58.255263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833bacd-6e11-4cf3-80df-37317adb7761 None admin] RBAC: Authorizing `identity:create_grant(project_id=1244700aec90459fbdbde9822e93a110, user_id=f84b5144054d4e158a838b4ae551996f, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:58.256909 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833bacd-6e11-4cf3-80df-37317adb7761 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:58.274078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 63/129] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:31:58 2022] PUT /identity/v3/projects/1244700aec90459fbdbde9822e93a110/users/f84b5144054d4e158a838b4ae551996f/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:31:58.274764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833bacd-6e11-4cf3-80df-37317adb7761 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:58.315328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a82dc-b6e1-4556-bc7a-50062c48a043 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:58.357660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af02df6-40da-4919-a399-435ddcd93865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:58.359211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af02df6-40da-4919-a399-435ddcd93865 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:58.360023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af02df6-40da-4919-a399-435ddcd93865 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:58.360492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af02df6-40da-4919-a399-435ddcd93865 None admin] PATH_INFO: `/v3/users/902e1680a1bd4607ad018968c6bfe87c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:58.362600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af02df6-40da-4919-a399-435ddcd93865 None admin] RBAC: Authorizing `identity:get_user(user_id=902e1680a1bd4607ad018968c6bfe87c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:58.364762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af02df6-40da-4919-a399-435ddcd93865 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:58.367288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 67/130] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:31:58 2022] GET /identity/v3/users/902e1680a1bd4607ad018968c6bfe87c => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:58.368334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af02df6-40da-4919-a399-435ddcd93865 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:58.397973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:58.425194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833bacd-6e11-4cf3-80df-37317adb7761 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:58.469970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:58.471687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071e2b3-cd53-482d-8a52-318979423ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:58.472940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071e2b3-cd53-482d-8a52-318979423ecd None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:58.473488 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071e2b3-cd53-482d-8a52-318979423ecd None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:58.473971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071e2b3-cd53-482d-8a52-318979423ecd None admin] PATH_INFO: `/v3/projects/050f7c5b157546cc884011378cf96ce1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:58.475826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071e2b3-cd53-482d-8a52-318979423ecd None admin] RBAC: Authorizing `identity:get_project(project_id=050f7c5b157546cc884011378cf96ce1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:58.478140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071e2b3-cd53-482d-8a52-318979423ecd None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:58.481345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 64/131] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:31:58 2022] GET /identity/v3/projects/050f7c5b157546cc884011378cf96ce1 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:58.482666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071e2b3-cd53-482d-8a52-318979423ecd None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:58.498926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af02df6-40da-4919-a399-435ddcd93865 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:58.537664 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c169004-d3d4-4b14-b1c7-58d824f13350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:58.538428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c169004-d3d4-4b14-b1c7-58d824f13350 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:58.539312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c169004-d3d4-4b14-b1c7-58d824f13350 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:58.539668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c169004-d3d4-4b14-b1c7-58d824f13350 None admin] PATH_INFO: `/v3/users/902e1680a1bd4607ad018968c6bfe87c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:58.541206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c169004-d3d4-4b14-b1c7-58d824f13350 None admin] RBAC: Authorizing `identity:get_user(user_id=902e1680a1bd4607ad018968c6bfe87c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:58.542860 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c169004-d3d4-4b14-b1c7-58d824f13350 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:58.544790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 68/132] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:31:58 2022] GET /identity/v3/users/902e1680a1bd4607ad018968c6bfe87c => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:58.550019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c169004-d3d4-4b14-b1c7-58d824f13350 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:58.621496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071e2b3-cd53-482d-8a52-318979423ecd None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:58.667711 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542ca60-6aeb-42bf-a369-5b0c41044e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:58.669227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542ca60-6aeb-42bf-a369-5b0c41044e2b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:58.670015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542ca60-6aeb-42bf-a369-5b0c41044e2b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:58.670504 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542ca60-6aeb-42bf-a369-5b0c41044e2b None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:58.672525 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542ca60-6aeb-42bf-a369-5b0c41044e2b None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:58.674470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542ca60-6aeb-42bf-a369-5b0c41044e2b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:58.677222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 65/133] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:31:58 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:58.678461 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542ca60-6aeb-42bf-a369-5b0c41044e2b None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:58.683546 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c169004-d3d4-4b14-b1c7-58d824f13350 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:58.702047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:58.726217 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b43135-d4c3-49f7-852c-34793f18263d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:58.727427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b43135-d4c3-49f7-852c-34793f18263d None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:58.727713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b43135-d4c3-49f7-852c-34793f18263d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:58.728048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b43135-d4c3-49f7-852c-34793f18263d None admin] PATH_INFO: `/v3/projects/050f7c5b157546cc884011378cf96ce1/users/902e1680a1bd4607ad018968c6bfe87c/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:58.730355 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b43135-d4c3-49f7-852c-34793f18263d None admin] RBAC: Authorizing `identity:create_grant(project_id=050f7c5b157546cc884011378cf96ce1, user_id=902e1680a1bd4607ad018968c6bfe87c, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:58.732450 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b43135-d4c3-49f7-852c-34793f18263d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:58.753784 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 69/134] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:31:58 2022] PUT /identity/v3/projects/050f7c5b157546cc884011378cf96ce1/users/902e1680a1bd4607ad018968c6bfe87c/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:31:58.754996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b43135-d4c3-49f7-852c-34793f18263d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:58.775592 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:58.900670 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542ca60-6aeb-42bf-a369-5b0c41044e2b None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:58.929428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b43135-d4c3-49f7-852c-34793f18263d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:58.990331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947badc-d5e3-4796-ab05-321f6c8fee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:58.991026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947badc-d5e3-4796-ab05-321f6c8fee4a None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:58.991313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947badc-d5e3-4796-ab05-321f6c8fee4a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:58.991594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947badc-d5e3-4796-ab05-321f6c8fee4a None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110/users/902e1680a1bd4607ad018968c6bfe87c/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:58.994968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947badc-d5e3-4796-ab05-321f6c8fee4a None admin] RBAC: Authorizing `identity:create_grant(project_id=1244700aec90459fbdbde9822e93a110, user_id=902e1680a1bd4607ad018968c6bfe87c, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:58.998343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947badc-d5e3-4796-ab05-321f6c8fee4a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:58.999522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3dae2-d3ff-441a-bc73-02d6a5498b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:59.000175 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3dae2-d3ff-441a-bc73-02d6a5498b21 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:59.000458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3dae2-d3ff-441a-bc73-02d6a5498b21 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:59.000742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3dae2-d3ff-441a-bc73-02d6a5498b21 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:59.002726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3dae2-d3ff-441a-bc73-02d6a5498b21 None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:59.004285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3dae2-d3ff-441a-bc73-02d6a5498b21 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:59.007725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 66/135] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:31:58 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:31:59.014277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3dae2-d3ff-441a-bc73-02d6a5498b21 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:59.032663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 70/136] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:31:58 2022] PUT /identity/v3/projects/1244700aec90459fbdbde9822e93a110/users/902e1680a1bd4607ad018968c6bfe87c/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:31:59.074818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05a357-c88c-40d1-9276-1c45d63c8a5d None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:59.075174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05a357-c88c-40d1-9276-1c45d63c8a5d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:59.075387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05a357-c88c-40d1-9276-1c45d63c8a5d None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:59.076880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 71/137] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:31:59.082612 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d93f1e-7e73-4812-b174-84b4c98f3f99 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:59.082914 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d93f1e-7e73-4812-b174-84b4c98f3f99 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:59.083173 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d93f1e-7e73-4812-b174-84b4c98f3f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:59.110795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-94d93f1e-7e73-4812-b174-84b4c98f3f99 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:59.112055 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-94d93f1e-7e73-4812-b174-84b4c98f3f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:59.170319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:59.205531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3dae2-d3ff-441a-bc73-02d6a5498b21 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:59.252949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:31:59.261503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 72/138] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:31:59.263661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca584e8a-9b0f-4b6d-a237-b4cfd39316ba None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:59.263901 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca584e8a-9b0f-4b6d-a237-b4cfd39316ba None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:59.264210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca584e8a-9b0f-4b6d-a237-b4cfd39316ba None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:59.267342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 73/139] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:31:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:31:59.273420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf01eb4-d553-4027-97a3-be209560ac5c None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:59.273767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf01eb4-d553-4027-97a3-be209560ac5c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:59.274024 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf01eb4-d553-4027-97a3-be209560ac5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:59.277602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea24bd-5398-4576-9584-b7c841736795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:59.278533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea24bd-5398-4576-9584-b7c841736795 None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:59.279009 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea24bd-5398-4576-9584-b7c841736795 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:59.279491 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea24bd-5398-4576-9584-b7c841736795 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110/users/902e1680a1bd4607ad018968c6bfe87c/roles/c0bf52b447564f769d707e96facb20a1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:59.284088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea24bd-5398-4576-9584-b7c841736795 None admin] RBAC: Authorizing `identity:create_grant(project_id=1244700aec90459fbdbde9822e93a110, user_id=902e1680a1bd4607ad018968c6bfe87c, role_id=c0bf52b447564f769d707e96facb20a1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:31:59.285818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea24bd-5398-4576-9584-b7c841736795 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:31:59.295117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-ecf01eb4-d553-4027-97a3-be209560ac5c None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:31:59.296493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ecf01eb4-d553-4027-97a3-be209560ac5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:31:59.302370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 67/140] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:31:59 2022] PUT /identity/v3/projects/1244700aec90459fbdbde9822e93a110/users/902e1680a1bd4607ad018968c6bfe87c/roles/c0bf52b447564f769d707e96facb20a1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:31:59.374177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 74/141] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:31:59.656311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea24bd-5398-4576-9584-b7c841736795 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:59.780882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea24bd-5398-4576-9584-b7c841736795 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:31:59.817120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca59ad-44e2-4d88-a499-b1b48130b383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:31:59.817829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca59ad-44e2-4d88-a499-b1b48130b383 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:31:59.818123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca59ad-44e2-4d88-a499-b1b48130b383 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:31:59.818403 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca59ad-44e2-4d88-a499-b1b48130b383 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:31:59.829660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca59ad-44e2-4d88-a499-b1b48130b383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-96ca59ad-44e2-4d88-a499-b1b48130b383 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:31:59.834923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:31:59.835806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:31:59.836557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 68/142] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:31:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:31:59.842048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf01eb4-d553-4027-97a3-be209560ac5c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:59.869519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca59ad-44e2-4d88-a499-b1b48130b383 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:31:59.873494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:00.036369 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca59ad-44e2-4d88-a499-b1b48130b383 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:00.040473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf01eb4-d553-4027-97a3-be209560ac5c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:00.072700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f796e-e645-4408-aa6e-975d415a49fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:00.074357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f796e-e645-4408-aa6e-975d415a49fb None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:00.074357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f796e-e645-4408-aa6e-975d415a49fb None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:00.074357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f796e-e645-4408-aa6e-975d415a49fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:00.075316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f796e-e645-4408-aa6e-975d415a49fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:00.076924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f796e-e645-4408-aa6e-975d415a49fb None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:00.078911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 69/143] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:31:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:00.084311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f796e-e645-4408-aa6e-975d415a49fb None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:00.098033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:00.106546 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3141f-1b7d-4f16-970c-38590ed50be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:00.107379 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3141f-1b7d-4f16-970c-38590ed50be1 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:00.107803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3141f-1b7d-4f16-970c-38590ed50be1 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:00.108276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3141f-1b7d-4f16-970c-38590ed50be1 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:00.109090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3141f-1b7d-4f16-970c-38590ed50be1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:00.109528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3141f-1b7d-4f16-970c-38590ed50be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:00.111153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3141f-1b7d-4f16-970c-38590ed50be1 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:00.130554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 75/144] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:31:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:00.139395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3141f-1b7d-4f16-970c-38590ed50be1 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:00.237203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:00.290768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3141f-1b7d-4f16-970c-38590ed50be1 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:00.319861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f796e-e645-4408-aa6e-975d415a49fb None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:00.379738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc18a07-a7d3-4930-b5e0-6afa0ab4b245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:00.380382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc18a07-a7d3-4930-b5e0-6afa0ab4b245 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:00.380661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc18a07-a7d3-4930-b5e0-6afa0ab4b245 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:00.380935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc18a07-a7d3-4930-b5e0-6afa0ab4b245 None admin] PATH_INFO: `/v3/users/258ae95275494c5c8bb8e98d920d2c97` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:00.386446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc18a07-a7d3-4930-b5e0-6afa0ab4b245 None admin] RBAC: Authorizing `identity:get_user(user_id=258ae95275494c5c8bb8e98d920d2c97)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:00.387921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc18a07-a7d3-4930-b5e0-6afa0ab4b245 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:00.390309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 76/145] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:00 2022] GET /identity/v3/users/258ae95275494c5c8bb8e98d920d2c97 => generated 289 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:00.396751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc18a07-a7d3-4930-b5e0-6afa0ab4b245 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:00.419253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce2c94-6c9a-4d46-bca1-93d8e09337d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:00.419871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce2c94-6c9a-4d46-bca1-93d8e09337d2 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:00.420155 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce2c94-6c9a-4d46-bca1-93d8e09337d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:00.420444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce2c94-6c9a-4d46-bca1-93d8e09337d2 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:00.421097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce2c94-6c9a-4d46-bca1-93d8e09337d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:00.422634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce2c94-6c9a-4d46-bca1-93d8e09337d2 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:00.464386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 70/146] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:32:00 2022] POST /identity/v3/users => generated 283 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:32:00.520884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:00.577023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc18a07-a7d3-4930-b5e0-6afa0ab4b245 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:00.629363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ec3a5-b211-4343-92f0-0cf02c162c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:00.630272 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ec3a5-b211-4343-92f0-0cf02c162c7e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:00.630694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ec3a5-b211-4343-92f0-0cf02c162c7e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:00.631188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ec3a5-b211-4343-92f0-0cf02c162c7e None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:00.634481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ec3a5-b211-4343-92f0-0cf02c162c7e None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:00.635975 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ec3a5-b211-4343-92f0-0cf02c162c7e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:00.637659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 77/147] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:00 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:00.643481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce2c94-6c9a-4d46-bca1-93d8e09337d2 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:00.785789 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce2c94-6c9a-4d46-bca1-93d8e09337d2 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:00.821294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:00.831233 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff7185-9f9c-43e4-8714-c30f789b1613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:00.831840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff7185-9f9c-43e4-8714-c30f789b1613 None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:00.832105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff7185-9f9c-43e4-8714-c30f789b1613 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:00.832379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff7185-9f9c-43e4-8714-c30f789b1613 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110/users/258ae95275494c5c8bb8e98d920d2c97/roles/015747495a144cb0976ad270d651b48f` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:00.834362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff7185-9f9c-43e4-8714-c30f789b1613 None admin] RBAC: Authorizing `identity:create_grant(project_id=1244700aec90459fbdbde9822e93a110, user_id=258ae95275494c5c8bb8e98d920d2c97, role_id=015747495a144cb0976ad270d651b48f)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:00.836714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff7185-9f9c-43e4-8714-c30f789b1613 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:00.860118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 71/148] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:32:00 2022] PUT /identity/v3/projects/1244700aec90459fbdbde9822e93a110/users/258ae95275494c5c8bb8e98d920d2c97/roles/015747495a144cb0976ad270d651b48f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:32:01.775554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ff96c-2847-423b-be75-0f4468bd84ea None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:01.776582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ff96c-2847-423b-be75-0f4468bd84ea None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:01.777064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ff96c-2847-423b-be75-0f4468bd84ea None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:01.778639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 78/149] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:01.785100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f03d0-cdd0-4894-a983-798240c25288 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:01.785552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f03d0-cdd0-4894-a983-798240c25288 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:01.785638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f03d0-cdd0-4894-a983-798240c25288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:01.810791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-929f03d0-cdd0-4894-a983-798240c25288 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:01.812001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-929f03d0-cdd0-4894-a983-798240c25288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:01.909622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 72/150] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:01.953900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee682f60-44c6-43c4-963d-6edcda32e406 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:01.953900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee682f60-44c6-43c4-963d-6edcda32e406 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:01.953900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee682f60-44c6-43c4-963d-6edcda32e406 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:01.953900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 79/151] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:01.959363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9811df4-96dd-428b-804c-38990ac13f58 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:01.959655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9811df4-96dd-428b-804c-38990ac13f58 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:01.959932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9811df4-96dd-428b-804c-38990ac13f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:01.983401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c9811df4-96dd-428b-804c-38990ac13f58 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:01.984596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c9811df4-96dd-428b-804c-38990ac13f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:02.076682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 73/152] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:02.098733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e338c-a822-45ae-9189-8648eb04272c None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:02.098733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e338c-a822-45ae-9189-8648eb04272c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:02.098733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e338c-a822-45ae-9189-8648eb04272c None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:02.100571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 80/153] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:02.106618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b4eba-fad0-48be-8546-9c9a2f8dfbf5 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:02.106913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b4eba-fad0-48be-8546-9c9a2f8dfbf5 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:02.107186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b4eba-fad0-48be-8546-9c9a2f8dfbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:02.129964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-853b4eba-fad0-48be-8546-9c9a2f8dfbf5 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:02.131156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-853b4eba-fad0-48be-8546-9c9a2f8dfbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:02.189136 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:02.218189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 74/154] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:02.358194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e338c-a822-45ae-9189-8648eb04272c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:02.468045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e338c-a822-45ae-9189-8648eb04272c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:02.499691 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:02.509148 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5722f8c-53e0-4722-b2e0-8a995b9ff1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:02.509798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5722f8c-53e0-4722-b2e0-8a995b9ff1d6 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:02.510079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5722f8c-53e0-4722-b2e0-8a995b9ff1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:02.510347 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5722f8c-53e0-4722-b2e0-8a995b9ff1d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:02.524081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5722f8c-53e0-4722-b2e0-8a995b9ff1d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-c5722f8c-53e0-4722-b2e0-8a995b9ff1d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:02.528983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:02.529938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:02.531159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 81/155] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:02.536445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b4eba-fad0-48be-8546-9c9a2f8dfbf5 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:02.622300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5722f8c-53e0-4722-b2e0-8a995b9ff1d6 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:02.686698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b4eba-fad0-48be-8546-9c9a2f8dfbf5 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:02.749709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b239a95-a9a6-4193-b350-d838d84c90b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:02.750718 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b239a95-a9a6-4193-b350-d838d84c90b3 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:02.751006 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b239a95-a9a6-4193-b350-d838d84c90b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:02.751284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b239a95-a9a6-4193-b350-d838d84c90b3 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:02.751962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b239a95-a9a6-4193-b350-d838d84c90b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:02.752267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b239a95-a9a6-4193-b350-d838d84c90b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:02.753658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b239a95-a9a6-4193-b350-d838d84c90b3 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:02.763898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 75/156] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:02.764726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b239a95-a9a6-4193-b350-d838d84c90b3 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:02.810353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5722f8c-53e0-4722-b2e0-8a995b9ff1d6 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:02.836361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:02.894440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8addad0-6c6d-4c71-b2df-1eb6c1445d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:02.895092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8addad0-6c6d-4c71-b2df-1eb6c1445d9f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:02.895367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8addad0-6c6d-4c71-b2df-1eb6c1445d9f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:02.895633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8addad0-6c6d-4c71-b2df-1eb6c1445d9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:02.924617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8addad0-6c6d-4c71-b2df-1eb6c1445d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-e8addad0-6c6d-4c71-b2df-1eb6c1445d9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:02.930194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:02.931222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:02.932057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 82/157] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:32:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:02.933138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa68872-20c1-49dc-91cc-f34983472b85 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:02.933138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa68872-20c1-49dc-91cc-f34983472b85 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:02.933352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa68872-20c1-49dc-91cc-f34983472b85 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:02.939046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 83/158] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:02.939640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa68872-20c1-49dc-91cc-f34983472b85 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:02.945201 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b239a95-a9a6-4193-b350-d838d84c90b3 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:03.019373 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec92552-2387-4d38-8ef7-7f3b2898b35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:03.020026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec92552-2387-4d38-8ef7-7f3b2898b35c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:03.020305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec92552-2387-4d38-8ef7-7f3b2898b35c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:03.020579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec92552-2387-4d38-8ef7-7f3b2898b35c None admin] PATH_INFO: `/v3/roles/member` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:03.046832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec92552-2387-4d38-8ef7-7f3b2898b35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-aec92552-2387-4d38-8ef7-7f3b2898b35c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:03.052683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:03.053523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:03.054962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 76/159] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:03.055591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec92552-2387-4d38-8ef7-7f3b2898b35c None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:03.076466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:03.177407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa68872-20c1-49dc-91cc-f34983472b85 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:03.264314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:03.267387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eadf0b-df7b-4aba-9474-b67c37dc1dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:03.268273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eadf0b-df7b-4aba-9474-b67c37dc1dc7 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:03.268397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec92552-2387-4d38-8ef7-7f3b2898b35c None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:03.268867 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eadf0b-df7b-4aba-9474-b67c37dc1dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:03.269322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eadf0b-df7b-4aba-9474-b67c37dc1dc7 None admin] PATH_INFO: `/v3/users/902e1680a1bd4607ad018968c6bfe87c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:03.274790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eadf0b-df7b-4aba-9474-b67c37dc1dc7 None admin] RBAC: Authorizing `identity:get_user(user_id=902e1680a1bd4607ad018968c6bfe87c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:03.276520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eadf0b-df7b-4aba-9474-b67c37dc1dc7 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:03.278700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 84/160] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:02 2022] GET /identity/v3/users/902e1680a1bd4607ad018968c6bfe87c => generated 275 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:03.280091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a75ff-d1cc-40a9-a2b2-b4e3135ff3c1 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:03.280588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a75ff-d1cc-40a9-a2b2-b4e3135ff3c1 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:03.281049 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a75ff-d1cc-40a9-a2b2-b4e3135ff3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:03.309384 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-de7a75ff-d1cc-40a9-a2b2-b4e3135ff3c1 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:03.310921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-de7a75ff-d1cc-40a9-a2b2-b4e3135ff3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:03.342843 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2187980-96c2-40eb-9981-a550b08a6cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:03.343472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2187980-96c2-40eb-9981-a550b08a6cc9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:03.343757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2187980-96c2-40eb-9981-a550b08a6cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:03.344163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2187980-96c2-40eb-9981-a550b08a6cc9 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:03.344987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2187980-96c2-40eb-9981-a550b08a6cc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:03.345399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2187980-96c2-40eb-9981-a550b08a6cc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:03.347046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2187980-96c2-40eb-9981-a550b08a6cc9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:03.366176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 77/161] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:03.367061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2187980-96c2-40eb-9981-a550b08a6cc9 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:03.393006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 85/162] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:03.394989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a75ff-d1cc-40a9-a2b2-b4e3135ff3c1 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:03.512876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:03.582646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2187980-96c2-40eb-9981-a550b08a6cc9 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:03.591009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a75ff-d1cc-40a9-a2b2-b4e3135ff3c1 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:03.645425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852b9e8-6349-4a84-89e6-b6ea56e13b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:03.646193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852b9e8-6349-4a84-89e6-b6ea56e13b39 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:03.646544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852b9e8-6349-4a84-89e6-b6ea56e13b39 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:03.646992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852b9e8-6349-4a84-89e6-b6ea56e13b39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:03.665710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888e092-c3b8-4771-80e0-8891796db1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:03.665710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888e092-c3b8-4771-80e0-8891796db1b1 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:03.665710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888e092-c3b8-4771-80e0-8891796db1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:03.665710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888e092-c3b8-4771-80e0-8891796db1b1 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:03.666733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888e092-c3b8-4771-80e0-8891796db1b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:03.667132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888e092-c3b8-4771-80e0-8891796db1b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:03.668628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888e092-c3b8-4771-80e0-8891796db1b1 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:03.684894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 78/163] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:03.686056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888e092-c3b8-4771-80e0-8891796db1b1 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:03.740840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852b9e8-6349-4a84-89e6-b6ea56e13b39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:03.742792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:03.743366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852b9e8-6349-4a84-89e6-b6ea56e13b39 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:03.799275 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:03.849526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 86/164] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:03.851655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852b9e8-6349-4a84-89e6-b6ea56e13b39 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:03.891172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888e092-c3b8-4771-80e0-8891796db1b1 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:03.950482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a90696-141b-492b-a7aa-0ca0a8072ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:03.951156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a90696-141b-492b-a7aa-0ca0a8072ea3 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:03.951667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a90696-141b-492b-a7aa-0ca0a8072ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:03.951782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a90696-141b-492b-a7aa-0ca0a8072ea3 None admin] PATH_INFO: `/v3/users/f84b5144054d4e158a838b4ae551996f` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:03.954508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a90696-141b-492b-a7aa-0ca0a8072ea3 None admin] RBAC: Authorizing `identity:get_user(user_id=f84b5144054d4e158a838b4ae551996f)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:03.956088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a90696-141b-492b-a7aa-0ca0a8072ea3 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:03.959098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 79/165] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:03 2022] GET /identity/v3/users/f84b5144054d4e158a838b4ae551996f => generated 247 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:03.959755 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a90696-141b-492b-a7aa-0ca0a8072ea3 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:03.962103 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852b9e8-6349-4a84-89e6-b6ea56e13b39 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:03.999290 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4896897-ac5b-4c74-8505-dc691eebaee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:04.000267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4896897-ac5b-4c74-8505-dc691eebaee8 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:04.000737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4896897-ac5b-4c74-8505-dc691eebaee8 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:04.001192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4896897-ac5b-4c74-8505-dc691eebaee8 None admin] PATH_INFO: `/v3/users/902e1680a1bd4607ad018968c6bfe87c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:04.002569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4896897-ac5b-4c74-8505-dc691eebaee8 None admin] RBAC: Authorizing `identity:get_user(user_id=902e1680a1bd4607ad018968c6bfe87c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:04.004313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4896897-ac5b-4c74-8505-dc691eebaee8 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:04.006599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 87/166] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:03 2022] GET /identity/v3/users/902e1680a1bd4607ad018968c6bfe87c => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:04.007722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4896897-ac5b-4c74-8505-dc691eebaee8 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:04.025231 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:04.114913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4896897-ac5b-4c74-8505-dc691eebaee8 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:04.132991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:04.153475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aeaefd-b419-4a35-a492-42f8502ae61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:04.154441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aeaefd-b419-4a35-a492-42f8502ae61c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:04.154909 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aeaefd-b419-4a35-a492-42f8502ae61c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:04.155391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aeaefd-b419-4a35-a492-42f8502ae61c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:04.155861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a90696-141b-492b-a7aa-0ca0a8072ea3 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:04.229795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aeaefd-b419-4a35-a492-42f8502ae61c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:04.232191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aeaefd-b419-4a35-a492-42f8502ae61c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:04.250073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:04.262187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89a1c9-780d-4f53-88f8-5a3ed1529b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:04.262849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89a1c9-780d-4f53-88f8-5a3ed1529b90 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:04.263179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89a1c9-780d-4f53-88f8-5a3ed1529b90 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:04.263399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89a1c9-780d-4f53-88f8-5a3ed1529b90 None admin] PATH_INFO: `/v3/projects/050f7c5b157546cc884011378cf96ce1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:04.266501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89a1c9-780d-4f53-88f8-5a3ed1529b90 None admin] RBAC: Authorizing `identity:get_project(project_id=050f7c5b157546cc884011378cf96ce1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:04.268021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89a1c9-780d-4f53-88f8-5a3ed1529b90 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:04.269670 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 80/167] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:03 2022] GET /identity/v3/projects/050f7c5b157546cc884011378cf96ce1 => generated 312 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:04.275406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89a1c9-780d-4f53-88f8-5a3ed1529b90 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:04.384526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89a1c9-780d-4f53-88f8-5a3ed1529b90 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:04.411382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 88/168] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:04.412247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aeaefd-b419-4a35-a492-42f8502ae61c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:04.436815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea77de2-0a68-43e1-8df0-3c419204006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:04.437429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea77de2-0a68-43e1-8df0-3c419204006d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:04.437731 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea77de2-0a68-43e1-8df0-3c419204006d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:04.438010 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea77de2-0a68-43e1-8df0-3c419204006d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:04.479934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:04.558682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea77de2-0a68-43e1-8df0-3c419204006d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:04.560188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea77de2-0a68-43e1-8df0-3c419204006d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:04.609444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aeaefd-b419-4a35-a492-42f8502ae61c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:04.630792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:04.681433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a90eb4-faca-46cc-8ee4-302dbb1aec4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:04.685472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a90eb4-faca-46cc-8ee4-302dbb1aec4e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:04.685999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a90eb4-faca-46cc-8ee4-302dbb1aec4e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:04.686481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a90eb4-faca-46cc-8ee4-302dbb1aec4e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:04.705465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a90eb4-faca-46cc-8ee4-302dbb1aec4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-33a90eb4-faca-46cc-8ee4-302dbb1aec4e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:04.711815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:04.712692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:04.713487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:04.713487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:04.713487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:04.713487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:04.713487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:04.713487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:04.713487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:04.713487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:04.713487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:04.713487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:04.713487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:04.713487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:04.722266 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 89/169] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:32:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:04.724489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a90eb4-faca-46cc-8ee4-302dbb1aec4e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:04.745988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 81/170] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:04.747159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea77de2-0a68-43e1-8df0-3c419204006d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:04.861255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:04.917461 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a90eb4-faca-46cc-8ee4-302dbb1aec4e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:04.940847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea77de2-0a68-43e1-8df0-3c419204006d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:04.963960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31589405-d94c-4ba3-8011-0c78a0f286e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:04.965122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31589405-d94c-4ba3-8011-0c78a0f286e7 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:04.966304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31589405-d94c-4ba3-8011-0c78a0f286e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:04.966946 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31589405-d94c-4ba3-8011-0c78a0f286e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:04.969185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31589405-d94c-4ba3-8011-0c78a0f286e7 None admin] RBAC: Adding query filter params (role.id=efddf5f5222644b18b6836117621adf8, scope.project.id=050f7c5b157546cc884011378cf96ce1, user.id=902e1680a1bd4607ad018968c6bfe87c) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:04.969716 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31589405-d94c-4ba3-8011-0c78a0f286e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:04.971575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31589405-d94c-4ba3-8011-0c78a0f286e7 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:04.979591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 90/171] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:04 2022] GET /identity/v3/role_assignments?user.id=902e1680a1bd4607ad018968c6bfe87c&scope.project.id=050f7c5b157546cc884011378cf96ce1&role.id=efddf5f5222644b18b6836117621adf8 => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:04.981177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2d1d9-b923-4a43-a503-5217e68bf6f2 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:04.981747 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2d1d9-b923-4a43-a503-5217e68bf6f2 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:04.982226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2d1d9-b923-4a43-a503-5217e68bf6f2 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:04.983733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 91/172] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:04.984627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2d1d9-b923-4a43-a503-5217e68bf6f2 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:05.033334 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c214b8-6b26-4c79-bb03-9c9fd10491e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:05.034292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c214b8-6b26-4c79-bb03-9c9fd10491e7 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:05.034727 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c214b8-6b26-4c79-bb03-9c9fd10491e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:05.035153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c214b8-6b26-4c79-bb03-9c9fd10491e7 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:05.036110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:05.036771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c214b8-6b26-4c79-bb03-9c9fd10491e7 None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:05.039102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c214b8-6b26-4c79-bb03-9c9fd10491e7 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:05.041209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 82/173] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:04 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:05.047110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c214b8-6b26-4c79-bb03-9c9fd10491e7 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:05.111767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2d1d9-b923-4a43-a503-5217e68bf6f2 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:05.150649 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf550a35-6740-4f87-9ebb-fd9340743953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:05.151270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf550a35-6740-4f87-9ebb-fd9340743953 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:05.151548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf550a35-6740-4f87-9ebb-fd9340743953 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:05.151816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf550a35-6740-4f87-9ebb-fd9340743953 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:05.152525 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf550a35-6740-4f87-9ebb-fd9340743953 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:05.152938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf550a35-6740-4f87-9ebb-fd9340743953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:05.154389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf550a35-6740-4f87-9ebb-fd9340743953 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:05.163342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 92/174] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:05.165046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404b63a-3703-454f-8503-8a97d96a2891 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:05.165333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404b63a-3703-454f-8503-8a97d96a2891 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:05.165638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404b63a-3703-454f-8503-8a97d96a2891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:05.185460 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-c404b63a-3703-454f-8503-8a97d96a2891 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:05.186614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c404b63a-3703-454f-8503-8a97d96a2891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:05.251581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c214b8-6b26-4c79-bb03-9c9fd10491e7 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:05.268151 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 93/175] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:05.269019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404b63a-3703-454f-8503-8a97d96a2891 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:05.300410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:05.306684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:05.314464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390c6a5-cbc5-4a73-9ed8-a713e48d655a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:05.315115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390c6a5-cbc5-4a73-9ed8-a713e48d655a None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:05.315398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390c6a5-cbc5-4a73-9ed8-a713e48d655a None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:05.315673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390c6a5-cbc5-4a73-9ed8-a713e48d655a None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:05.318476 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390c6a5-cbc5-4a73-9ed8-a713e48d655a None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:05.320097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390c6a5-cbc5-4a73-9ed8-a713e48d655a None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:05.322100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 83/176] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:05 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:05.322890 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390c6a5-cbc5-4a73-9ed8-a713e48d655a None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:05.428680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404b63a-3703-454f-8503-8a97d96a2891 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:05.438581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390c6a5-cbc5-4a73-9ed8-a713e48d655a None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:05.481672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a59be1-ed6a-4d5e-8720-f9fb893712f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:05.482640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a59be1-ed6a-4d5e-8720-f9fb893712f0 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:05.483156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a59be1-ed6a-4d5e-8720-f9fb893712f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:05.483678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a59be1-ed6a-4d5e-8720-f9fb893712f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:05.484595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a59be1-ed6a-4d5e-8720-f9fb893712f0 None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=1244700aec90459fbdbde9822e93a110, user.id=f84b5144054d4e158a838b4ae551996f) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:05.485104 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a59be1-ed6a-4d5e-8720-f9fb893712f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:05.486785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a59be1-ed6a-4d5e-8720-f9fb893712f0 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:05.486981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f1624-acfd-4724-8e30-755607a6669d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:05.487617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f1624-acfd-4724-8e30-755607a6669d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:05.487894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f1624-acfd-4724-8e30-755607a6669d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:05.488165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f1624-acfd-4724-8e30-755607a6669d None admin] PATH_INFO: `/v3/users/902e1680a1bd4607ad018968c6bfe87c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:05.490947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f1624-acfd-4724-8e30-755607a6669d None admin] RBAC: Authorizing `identity:get_user(user_id=902e1680a1bd4607ad018968c6bfe87c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:05.492944 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f1624-acfd-4724-8e30-755607a6669d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:05.494705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 84/177] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:05 2022] GET /identity/v3/users/902e1680a1bd4607ad018968c6bfe87c => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:05.495324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f1624-acfd-4724-8e30-755607a6669d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:05.495612 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 94/178] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:05 2022] GET /identity/v3/role_assignments?user.id=f84b5144054d4e158a838b4ae551996f&scope.project.id=1244700aec90459fbdbde9822e93a110&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:05.497035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32affca-234e-4633-92ee-0c6f8c6c5341 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:05.497522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32affca-234e-4633-92ee-0c6f8c6c5341 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:05.498068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32affca-234e-4633-92ee-0c6f8c6c5341 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:05.499660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 95/179] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:05.505716 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd678e9-00f9-4f5f-9943-fdf3d0abcade None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:05.506547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd678e9-00f9-4f5f-9943-fdf3d0abcade None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:05.507117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd678e9-00f9-4f5f-9943-fdf3d0abcade None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:05.509452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 96/180] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:05.510412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd678e9-00f9-4f5f-9943-fdf3d0abcade None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:05.574171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:05.625999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f1624-acfd-4724-8e30-755607a6669d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:05.695528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f9774-5e7b-477e-bc77-54a2845347bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:05.696029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f9774-5e7b-477e-bc77-54a2845347bf None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:05.696311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f9774-5e7b-477e-bc77-54a2845347bf None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:05.696589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f9774-5e7b-477e-bc77-54a2845347bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:05.696723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:05.697330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f9774-5e7b-477e-bc77-54a2845347bf None admin] RBAC: Adding query filter params (role.id=efddf5f5222644b18b6836117621adf8, scope.project.id=1244700aec90459fbdbde9822e93a110, user.id=902e1680a1bd4607ad018968c6bfe87c) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:05.697641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f9774-5e7b-477e-bc77-54a2845347bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:05.699343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f9774-5e7b-477e-bc77-54a2845347bf None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:05.707357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd678e9-00f9-4f5f-9943-fdf3d0abcade None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:05.717802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 85/181] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:05 2022] GET /identity/v3/role_assignments?user.id=902e1680a1bd4607ad018968c6bfe87c&scope.project.id=1244700aec90459fbdbde9822e93a110&role.id=efddf5f5222644b18b6836117621adf8 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:05.722289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200d425-2e95-4e86-af02-c56ee8839341 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:05.722869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200d425-2e95-4e86-af02-c56ee8839341 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:05.723358 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200d425-2e95-4e86-af02-c56ee8839341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:05.751056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-0200d425-2e95-4e86-af02-c56ee8839341 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:05.752338 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0200d425-2e95-4e86-af02-c56ee8839341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:05.773447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09a875-6c47-4a38-a506-bd0e532c68ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:05.774493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a875-6c47-4a38-a506-bd0e532c68ab None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:05.774994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a875-6c47-4a38-a506-bd0e532c68ab None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:05.775438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a875-6c47-4a38-a506-bd0e532c68ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:05.836882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 86/182] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:05.839124 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3679479-a385-4e6f-95d1-0fd010fc26db None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:05.839414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3679479-a385-4e6f-95d1-0fd010fc26db None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:05.839688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3679479-a385-4e6f-95d1-0fd010fc26db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:05.863948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-b3679479-a385-4e6f-95d1-0fd010fc26db None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:05.865255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b3679479-a385-4e6f-95d1-0fd010fc26db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:05.869370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09a875-6c47-4a38-a506-bd0e532c68ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:05.871114 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09a875-6c47-4a38-a506-bd0e532c68ab None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:05.935387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:05.951897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 87/183] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:05.952570 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3679479-a385-4e6f-95d1-0fd010fc26db None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:05.995327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:06.005873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 97/184] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:06.007858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19482148-a50d-463a-a72c-6b7f252c3c07 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:06.008335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19482148-a50d-463a-a72c-6b7f252c3c07 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:06.008808 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19482148-a50d-463a-a72c-6b7f252c3c07 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:06.010702 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 98/185] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:06.011611 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19482148-a50d-463a-a72c-6b7f252c3c07 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:06.063614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3679479-a385-4e6f-95d1-0fd010fc26db None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:06.100835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8a9d9-b5ea-4fbf-992a-ef79faeb160f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:06.101503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8a9d9-b5ea-4fbf-992a-ef79faeb160f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:06.102315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8a9d9-b5ea-4fbf-992a-ef79faeb160f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:06.102588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8a9d9-b5ea-4fbf-992a-ef79faeb160f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:06.113907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8a9d9-b5ea-4fbf-992a-ef79faeb160f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-f4d8a9d9-b5ea-4fbf-992a-ef79faeb160f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:06.119603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:06.120463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:06.121255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 88/186] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:06.123293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f716c2-d092-4bd7-b012-57cac0976e0e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:06.123586 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f716c2-d092-4bd7-b012-57cac0976e0e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:06.123925 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f716c2-d092-4bd7-b012-57cac0976e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:06.136730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19482148-a50d-463a-a72c-6b7f252c3c07 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:06.142489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-b3f716c2-d092-4bd7-b012-57cac0976e0e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:06.143691 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b3f716c2-d092-4bd7-b012-57cac0976e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:06.176385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cdf83-7aee-48ed-ad88-344df2501c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:06.177346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cdf83-7aee-48ed-ad88-344df2501c11 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:06.177861 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cdf83-7aee-48ed-ad88-344df2501c11 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:06.178427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cdf83-7aee-48ed-ad88-344df2501c11 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:06.180022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cdf83-7aee-48ed-ad88-344df2501c11 None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:06.181914 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cdf83-7aee-48ed-ad88-344df2501c11 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:06.183728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 99/187] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:06 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:06.184682 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cdf83-7aee-48ed-ad88-344df2501c11 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:06.189558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:06.212566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 89/188] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:06.213320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f716c2-d092-4bd7-b012-57cac0976e0e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:06.256703 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:06.305408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cdf83-7aee-48ed-ad88-344df2501c11 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:06.340634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd7c11-ca4a-4b3b-8ec9-085a2954d309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:06.341500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd7c11-ca4a-4b3b-8ec9-085a2954d309 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:06.341811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd7c11-ca4a-4b3b-8ec9-085a2954d309 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:06.342083 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd7c11-ca4a-4b3b-8ec9-085a2954d309 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:06.342901 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd7c11-ca4a-4b3b-8ec9-085a2954d309 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:06.343512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd7c11-ca4a-4b3b-8ec9-085a2954d309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:06.345155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd7c11-ca4a-4b3b-8ec9-085a2954d309 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:06.345725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f716c2-d092-4bd7-b012-57cac0976e0e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:06.360652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 100/189] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:06.360652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd7c11-ca4a-4b3b-8ec9-085a2954d309 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:06.381321 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d708345-52a6-4d64-9b28-db2b8275d7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:06.381981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d708345-52a6-4d64-9b28-db2b8275d7d5 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:06.382263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d708345-52a6-4d64-9b28-db2b8275d7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:06.382558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d708345-52a6-4d64-9b28-db2b8275d7d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:06.394392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d708345-52a6-4d64-9b28-db2b8275d7d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-5d708345-52a6-4d64-9b28-db2b8275d7d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:06.400357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:06.401251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:06.402050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 90/190] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:32:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:06.406780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d708345-52a6-4d64-9b28-db2b8275d7d5 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:06.451325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:06.468467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd7c11-ca4a-4b3b-8ec9-085a2954d309 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:06.492102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:06.503471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6602d8-0f28-4e68-9ca3-196df9a83c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:06.504080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6602d8-0f28-4e68-9ca3-196df9a83c6c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:06.504366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6602d8-0f28-4e68-9ca3-196df9a83c6c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:06.504642 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6602d8-0f28-4e68-9ca3-196df9a83c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:06.505376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6602d8-0f28-4e68-9ca3-196df9a83c6c None admin] RBAC: Adding query filter params (role.id=c0bf52b447564f769d707e96facb20a1, scope.project.id=1244700aec90459fbdbde9822e93a110, user.id=902e1680a1bd4607ad018968c6bfe87c) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:06.505710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6602d8-0f28-4e68-9ca3-196df9a83c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:06.507161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6602d8-0f28-4e68-9ca3-196df9a83c6c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:06.512908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 101/191] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:06 2022] GET /identity/v3/role_assignments?user.id=902e1680a1bd4607ad018968c6bfe87c&scope.project.id=1244700aec90459fbdbde9822e93a110&role.id=c0bf52b447564f769d707e96facb20a1 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:06.513670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6602d8-0f28-4e68-9ca3-196df9a83c6c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:06.515759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d708345-52a6-4d64-9b28-db2b8275d7d5 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:06.554040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd2be4-6917-47df-ba9c-e04455059410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:06.555029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd2be4-6917-47df-ba9c-e04455059410 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:06.555445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd2be4-6917-47df-ba9c-e04455059410 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:06.555864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd2be4-6917-47df-ba9c-e04455059410 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:06.557391 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd2be4-6917-47df-ba9c-e04455059410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:06.559066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd2be4-6917-47df-ba9c-e04455059410 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:06.561154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 91/192] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:06.562133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd2be4-6917-47df-ba9c-e04455059410 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:06.628178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6602d8-0f28-4e68-9ca3-196df9a83c6c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:06.670848 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677681e5-22d3-4672-acde-b1cb5cf735fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:06.671588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677681e5-22d3-4672-acde-b1cb5cf735fd None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:06.671871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677681e5-22d3-4672-acde-b1cb5cf735fd None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:06.672144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677681e5-22d3-4672-acde-b1cb5cf735fd None admin] PATH_INFO: `/v3/users/258ae95275494c5c8bb8e98d920d2c97` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:06.673327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677681e5-22d3-4672-acde-b1cb5cf735fd None admin] RBAC: Authorizing `identity:get_user(user_id=258ae95275494c5c8bb8e98d920d2c97)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:06.674854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677681e5-22d3-4672-acde-b1cb5cf735fd None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:06.676475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 102/193] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:06 2022] GET /identity/v3/users/258ae95275494c5c8bb8e98d920d2c97 => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:06.677243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677681e5-22d3-4672-acde-b1cb5cf735fd None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:06.689258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd2be4-6917-47df-ba9c-e04455059410 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:06.697714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:06.727532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633a10f-a765-499b-9535-649cc0a6a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:06.728063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633a10f-a765-499b-9535-649cc0a6a391 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:06.728341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633a10f-a765-499b-9535-649cc0a6a391 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:06.729011 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633a10f-a765-499b-9535-649cc0a6a391 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:06.729575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633a10f-a765-499b-9535-649cc0a6a391 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:06.729962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633a10f-a765-499b-9535-649cc0a6a391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:06.731395 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633a10f-a765-499b-9535-649cc0a6a391 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:06.742412 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 92/194] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:06.743061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633a10f-a765-499b-9535-649cc0a6a391 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:06.744740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:06.796218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677681e5-22d3-4672-acde-b1cb5cf735fd None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:06.828706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316847a-75b1-495d-9eff-b77d18c65d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:06.829323 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316847a-75b1-495d-9eff-b77d18c65d82 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:06.829600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316847a-75b1-495d-9eff-b77d18c65d82 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:06.829904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316847a-75b1-495d-9eff-b77d18c65d82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:06.839797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316847a-75b1-495d-9eff-b77d18c65d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:06.844615 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633a10f-a765-499b-9535-649cc0a6a391 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-a316847a-75b1-495d-9eff-b77d18c65d82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:06.844930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:06.845902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:06.846697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 103/195] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:32:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:06.847325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316847a-75b1-495d-9eff-b77d18c65d82 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:06.880365 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa51eb1-36e3-43de-a558-66ca81c9182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:06.880962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa51eb1-36e3-43de-a558-66ca81c9182e None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:06.881238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa51eb1-36e3-43de-a558-66ca81c9182e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:06.881507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa51eb1-36e3-43de-a558-66ca81c9182e None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:06.882216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa51eb1-36e3-43de-a558-66ca81c9182e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:06.884190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa51eb1-36e3-43de-a558-66ca81c9182e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:06.904831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 93/196] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:32:06 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:32:06.905650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa51eb1-36e3-43de-a558-66ca81c9182e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:06.934473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:06.969004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316847a-75b1-495d-9eff-b77d18c65d82 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:06.987887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:07.009965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c3c8b-3865-4a95-9c46-8091a2c78998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:07.010647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c3c8b-3865-4a95-9c46-8091a2c78998 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:07.010883 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c3c8b-3865-4a95-9c46-8091a2c78998 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:07.011683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c3c8b-3865-4a95-9c46-8091a2c78998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:07.024097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa51eb1-36e3-43de-a558-66ca81c9182e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:07.068106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b5e61-678d-46c0-8e1d-344fcd1f680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:07.068757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5e61-678d-46c0-8e1d-344fcd1f680c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:07.069079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5e61-678d-46c0-8e1d-344fcd1f680c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:07.069998 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b5e61-678d-46c0-8e1d-344fcd1f680c None admin] PATH_INFO: `/v3/users/b28acfc573784d58bbeb74a5c8057c22` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:07.087335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b5e61-678d-46c0-8e1d-344fcd1f680c None admin] RBAC: Authorizing `identity:get_user(user_id=b28acfc573784d58bbeb74a5c8057c22)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:07.089353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b5e61-678d-46c0-8e1d-344fcd1f680c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:07.089460 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c3c8b-3865-4a95-9c46-8091a2c78998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:07.090582 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 94/197] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:06 2022] GET /identity/v3/users/b28acfc573784d58bbeb74a5c8057c22 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:07.090687 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c3c8b-3865-4a95-9c46-8091a2c78998 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:07.094481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b5e61-678d-46c0-8e1d-344fcd1f680c None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:07.192524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:07.210350 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 104/198] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:07.211996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c3c8b-3865-4a95-9c46-8091a2c78998 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:07.219176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b5e61-678d-46c0-8e1d-344fcd1f680c None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:07.234185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:07.263693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e452215-696b-4101-bbd8-5318878672b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:07.264231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e452215-696b-4101-bbd8-5318878672b8 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:07.264507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e452215-696b-4101-bbd8-5318878672b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:07.264775 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e452215-696b-4101-bbd8-5318878672b8 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:07.265431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e452215-696b-4101-bbd8-5318878672b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:07.265757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e452215-696b-4101-bbd8-5318878672b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:07.267129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e452215-696b-4101-bbd8-5318878672b8 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:07.276505 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 95/199] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:07.277318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e452215-696b-4101-bbd8-5318878672b8 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:07.328155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c3c8b-3865-4a95-9c46-8091a2c78998 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:07.366796 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aad0b6-fd50-4d64-97c5-6f806abe8ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:07.367895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aad0b6-fd50-4d64-97c5-6f806abe8ac9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:07.368174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aad0b6-fd50-4d64-97c5-6f806abe8ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:07.368493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aad0b6-fd50-4d64-97c5-6f806abe8ac9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:07.412305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e452215-696b-4101-bbd8-5318878672b8 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:07.447262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aad0b6-fd50-4d64-97c5-6f806abe8ac9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:07.450033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aad0b6-fd50-4d64-97c5-6f806abe8ac9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:07.459030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6e938-3216-42a1-8172-f62471881d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:07.459664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6e938-3216-42a1-8172-f62471881d4d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:07.459959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6e938-3216-42a1-8172-f62471881d4d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:07.460294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6e938-3216-42a1-8172-f62471881d4d None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:07.461805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6e938-3216-42a1-8172-f62471881d4d None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:07.463073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6e938-3216-42a1-8172-f62471881d4d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:07.465107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 96/200] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:07 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:07.465830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6e938-3216-42a1-8172-f62471881d4d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:07.473789 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:07.484684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:07.572558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 105/201] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:07.573770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aad0b6-fd50-4d64-97c5-6f806abe8ac9 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:07.599913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6e938-3216-42a1-8172-f62471881d4d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:07.635536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645b2df-e488-4394-8588-389f945712cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:07.636040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645b2df-e488-4394-8588-389f945712cc None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:07.636319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645b2df-e488-4394-8588-389f945712cc None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:07.636593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645b2df-e488-4394-8588-389f945712cc None admin] PATH_INFO: `/v3/users/b28acfc573784d58bbeb74a5c8057c22` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:07.637801 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645b2df-e488-4394-8588-389f945712cc None admin] RBAC: Authorizing `identity:get_user(user_id=b28acfc573784d58bbeb74a5c8057c22)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:07.639252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645b2df-e488-4394-8588-389f945712cc None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:07.640987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 97/202] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:07 2022] GET /identity/v3/users/b28acfc573784d58bbeb74a5c8057c22 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:07.641624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645b2df-e488-4394-8588-389f945712cc None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:07.696634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aad0b6-fd50-4d64-97c5-6f806abe8ac9 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:07.714082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:07.732160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26514ef-6fb3-4be1-8d07-6d53d63e899f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:07.733014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26514ef-6fb3-4be1-8d07-6d53d63e899f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:07.733134 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26514ef-6fb3-4be1-8d07-6d53d63e899f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:07.733332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26514ef-6fb3-4be1-8d07-6d53d63e899f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:07.734084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26514ef-6fb3-4be1-8d07-6d53d63e899f None admin] RBAC: Adding query filter params (role.id=015747495a144cb0976ad270d651b48f, scope.project.id=1244700aec90459fbdbde9822e93a110, user.id=258ae95275494c5c8bb8e98d920d2c97) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:07.734393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26514ef-6fb3-4be1-8d07-6d53d63e899f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:07.735953 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26514ef-6fb3-4be1-8d07-6d53d63e899f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:07.741929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 106/203] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:07 2022] GET /identity/v3/role_assignments?user.id=258ae95275494c5c8bb8e98d920d2c97&scope.project.id=1244700aec90459fbdbde9822e93a110&role.id=015747495a144cb0976ad270d651b48f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:07.742728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26514ef-6fb3-4be1-8d07-6d53d63e899f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:07.752585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:07.781815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645b2df-e488-4394-8588-389f945712cc None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:07.820739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40f677-6dc1-43ff-a43d-01968056f6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:07.821390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40f677-6dc1-43ff-a43d-01968056f6ff None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:07.821739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40f677-6dc1-43ff-a43d-01968056f6ff None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:07.822070 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40f677-6dc1-43ff-a43d-01968056f6ff None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:07.823424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40f677-6dc1-43ff-a43d-01968056f6ff None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:07.825140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40f677-6dc1-43ff-a43d-01968056f6ff None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:07.826877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 98/204] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:07 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:07.832655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40f677-6dc1-43ff-a43d-01968056f6ff None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:07.877385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26514ef-6fb3-4be1-8d07-6d53d63e899f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:07.926108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6761ca-919a-4714-b408-bba74791bc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:07.927390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6761ca-919a-4714-b408-bba74791bc98 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:07.928331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6761ca-919a-4714-b408-bba74791bc98 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:07.929065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6761ca-919a-4714-b408-bba74791bc98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:07.960023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40f677-6dc1-43ff-a43d-01968056f6ff None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:08.032389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3029cbe-a578-4d6f-a845-2c7bf3b2f21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:08.033042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029cbe-a578-4d6f-a845-2c7bf3b2f21e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:08.033265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029cbe-a578-4d6f-a845-2c7bf3b2f21e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:08.033584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3029cbe-a578-4d6f-a845-2c7bf3b2f21e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:08.035281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3029cbe-a578-4d6f-a845-2c7bf3b2f21e None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=e2499b13100b44bb8d6160b53bc3747c, user.id=b28acfc573784d58bbeb74a5c8057c22) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:08.035281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3029cbe-a578-4d6f-a845-2c7bf3b2f21e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:08.036152 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3029cbe-a578-4d6f-a845-2c7bf3b2f21e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:08.039065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:08.047126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 99/205] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:07 2022] GET /identity/v3/role_assignments?user.id=b28acfc573784d58bbeb74a5c8057c22&scope.project.id=e2499b13100b44bb8d6160b53bc3747c&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:08.082347 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6761ca-919a-4714-b408-bba74791bc98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:08.083771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6761ca-919a-4714-b408-bba74791bc98 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:08.246553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 107/206] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:08.251382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3029cbe-a578-4d6f-a845-2c7bf3b2f21e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:08.269729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:08.362283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3029cbe-a578-4d6f-a845-2c7bf3b2f21e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:08.404544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ace02d-5107-47ca-9c40-33111200fb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:08.405567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ace02d-5107-47ca-9c40-33111200fb24 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:08.406079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ace02d-5107-47ca-9c40-33111200fb24 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:08.406569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ace02d-5107-47ca-9c40-33111200fb24 None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:08.407930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ace02d-5107-47ca-9c40-33111200fb24 None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:08.409591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ace02d-5107-47ca-9c40-33111200fb24 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:08.411861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 100/207] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:08 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:08.418172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6761ca-919a-4714-b408-bba74791bc98 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:08.505978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:08.527760 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6761ca-919a-4714-b408-bba74791bc98 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:08.563091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446afd6b-b1a0-4f27-b02b-4f88f7fd0c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:08.563696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446afd6b-b1a0-4f27-b02b-4f88f7fd0c58 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:08.563975 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446afd6b-b1a0-4f27-b02b-4f88f7fd0c58 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:08.564247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446afd6b-b1a0-4f27-b02b-4f88f7fd0c58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:08.564971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446afd6b-b1a0-4f27-b02b-4f88f7fd0c58 None admin] RBAC: Adding query filter params (role.id=c0bf52b447564f769d707e96facb20a1, scope.project.id=e2499b13100b44bb8d6160b53bc3747c, user.id=b28acfc573784d58bbeb74a5c8057c22) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:08.565278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446afd6b-b1a0-4f27-b02b-4f88f7fd0c58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:08.566740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446afd6b-b1a0-4f27-b02b-4f88f7fd0c58 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:08.572812 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 108/208] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:08 2022] GET /identity/v3/role_assignments?user.id=b28acfc573784d58bbeb74a5c8057c22&scope.project.id=e2499b13100b44bb8d6160b53bc3747c&role.id=c0bf52b447564f769d707e96facb20a1 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:09.361220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f3fc5-d518-4b2b-92d0-9e1baa0c2c2a None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:09.361782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f3fc5-d518-4b2b-92d0-9e1baa0c2c2a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:09.361886 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f3fc5-d518-4b2b-92d0-9e1baa0c2c2a None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:09.363803 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 101/209] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:09.364842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75b090-8a63-4539-b165-b21e78bc3550 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:09.365299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75b090-8a63-4539-b165-b21e78bc3550 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:09.365457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75b090-8a63-4539-b165-b21e78bc3550 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:09.367367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 109/210] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:32:09.370027 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08b933-fad9-486d-9e99-2f4825a108de None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:09.370405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08b933-fad9-486d-9e99-2f4825a108de None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:09.370777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08b933-fad9-486d-9e99-2f4825a108de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:09.372378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c8e8f-aacf-46b7-ba31-0b5df81ca5f0 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:09.372661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c8e8f-aacf-46b7-ba31-0b5df81ca5f0 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:09.372931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c8e8f-aacf-46b7-ba31-0b5df81ca5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:09.390662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-7c08b933-fad9-486d-9e99-2f4825a108de None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:09.391296 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-1f1c8e8f-aacf-46b7-ba31-0b5df81ca5f0 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:09.392045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7c08b933-fad9-486d-9e99-2f4825a108de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:09.392592 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1f1c8e8f-aacf-46b7-ba31-0b5df81ca5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:09.468489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 110/211] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:09.475518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 102/212] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:09.743705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08b933-fad9-486d-9e99-2f4825a108de None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:09.743897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c8e8f-aacf-46b7-ba31-0b5df81ca5f0 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:09.758837 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:09.844069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:09.861245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08b933-fad9-486d-9e99-2f4825a108de None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:09.880324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c8e8f-aacf-46b7-ba31-0b5df81ca5f0 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:09.900186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ac7f5-5b34-4269-b4b6-3417f809cb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:09.901043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ac7f5-5b34-4269-b4b6-3417f809cb28 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:09.901511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ac7f5-5b34-4269-b4b6-3417f809cb28 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:09.901968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ac7f5-5b34-4269-b4b6-3417f809cb28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:09.913353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ac7f5-5b34-4269-b4b6-3417f809cb28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-d70ac7f5-5b34-4269-b4b6-3417f809cb28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:09.919305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:09.920420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:09.921197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:09.921197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:09.921197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:09.921197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:09.921197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:09.921197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:09.921197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:09.921197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:09.921197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:09.921197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:09.921197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:09.921197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:09.921841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c524a-fd7e-4a79-90c0-6496019a5a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:09.922223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c524a-fd7e-4a79-90c0-6496019a5a15 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:09.922681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c524a-fd7e-4a79-90c0-6496019a5a15 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:09.923017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c524a-fd7e-4a79-90c0-6496019a5a15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:09.924271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c524a-fd7e-4a79-90c0-6496019a5a15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:09.924468 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 103/213] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:09.927049 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c524a-fd7e-4a79-90c0-6496019a5a15 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:09.929717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 111/214] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:09.930832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ac7f5-5b34-4269-b4b6-3417f809cb28 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:09.934917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c524a-fd7e-4a79-90c0-6496019a5a15 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:10.008720 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:10.045001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ac7f5-5b34-4269-b4b6-3417f809cb28 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:10.067711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c524a-fd7e-4a79-90c0-6496019a5a15 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:10.084760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcd75e-d8ed-4302-8887-a470e99a3492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:10.086027 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcd75e-d8ed-4302-8887-a470e99a3492 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:10.086381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcd75e-d8ed-4302-8887-a470e99a3492 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:10.086726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcd75e-d8ed-4302-8887-a470e99a3492 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:10.087457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcd75e-d8ed-4302-8887-a470e99a3492 None admin] RBAC: Adding query filter params (name=member) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:10.087831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcd75e-d8ed-4302-8887-a470e99a3492 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:10.089345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfcd75e-d8ed-4302-8887-a470e99a3492 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:10.103990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 104/215] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:10.105859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcd75e-d8ed-4302-8887-a470e99a3492 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:10.115381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61407242-e74f-4f01-b709-eb03f3ac5ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:10.116289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61407242-e74f-4f01-b709-eb03f3ac5ef0 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:10.117305 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61407242-e74f-4f01-b709-eb03f3ac5ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:10.118209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61407242-e74f-4f01-b709-eb03f3ac5ef0 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:10.119092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61407242-e74f-4f01-b709-eb03f3ac5ef0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:10.121262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61407242-e74f-4f01-b709-eb03f3ac5ef0 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:10.150164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:10.160736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 112/216] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:32:09 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:32:10.233640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcd75e-d8ed-4302-8887-a470e99a3492 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:10.269079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:10.276051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e419b-8930-4730-9869-73b4b2de7e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:10.276728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e419b-8930-4730-9869-73b4b2de7e87 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:10.277011 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e419b-8930-4730-9869-73b4b2de7e87 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:10.277292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e419b-8930-4730-9869-73b4b2de7e87 None admin] PATH_INFO: `/v3/users/e0f4af2471dd4ebf9adcf2b62a178e75` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:10.294016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e419b-8930-4730-9869-73b4b2de7e87 None admin] RBAC: Authorizing `identity:get_user(user_id=e0f4af2471dd4ebf9adcf2b62a178e75)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:10.295535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e419b-8930-4730-9869-73b4b2de7e87 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:10.297642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 105/217] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:10 2022] GET /identity/v3/users/e0f4af2471dd4ebf9adcf2b62a178e75 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:10.303245 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61407242-e74f-4f01-b709-eb03f3ac5ef0 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:10.322278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81640cf-01b6-4815-a2f5-e0600092102b None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:10.322940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81640cf-01b6-4815-a2f5-e0600092102b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:10.323429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81640cf-01b6-4815-a2f5-e0600092102b None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:10.325189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 106/218] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:32:10.331494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f164b61-9147-4968-8f05-1f4318fcb49a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:10.331788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f164b61-9147-4968-8f05-1f4318fcb49a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:10.332039 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f164b61-9147-4968-8f05-1f4318fcb49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:10.352948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-0f164b61-9147-4968-8f05-1f4318fcb49a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:10.354694 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0f164b61-9147-4968-8f05-1f4318fcb49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:10.443503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61407242-e74f-4f01-b709-eb03f3ac5ef0 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:10.452938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 107/219] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:10.489317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae8bd3-5540-416b-9517-7ab94f5b0cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:10.490336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae8bd3-5540-416b-9517-7ab94f5b0cf9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:10.490816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae8bd3-5540-416b-9517-7ab94f5b0cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:10.491285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae8bd3-5540-416b-9517-7ab94f5b0cf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:10.560672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:10.564803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae8bd3-5540-416b-9517-7ab94f5b0cf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:10.566752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae8bd3-5540-416b-9517-7ab94f5b0cf9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:10.690919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 113/220] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:10.697365 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f164b61-9147-4968-8f05-1f4318fcb49a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:10.742206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae8bd3-5540-416b-9517-7ab94f5b0cf9 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:10.819257 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f164b61-9147-4968-8f05-1f4318fcb49a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:10.850123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:10.858331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c9690-38f0-433f-8160-90bdcd32a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:10.858599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:10.859369 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c9690-38f0-433f-8160-90bdcd32a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:10.859688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c9690-38f0-433f-8160-90bdcd32a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:10.860520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c9690-38f0-433f-8160-90bdcd32a8c7 None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:10.862630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c9690-38f0-433f-8160-90bdcd32a8c7 None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:10.863784 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae8bd3-5540-416b-9517-7ab94f5b0cf9 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:10.864096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c9690-38f0-433f-8160-90bdcd32a8c7 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:10.866241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 108/221] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:10 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:10.868481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6743e-2493-48f5-9ca5-f48da3fffe24 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:10.868759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6743e-2493-48f5-9ca5-f48da3fffe24 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:10.869029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6743e-2493-48f5-9ca5-f48da3fffe24 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:10.870260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 109/222] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:10.872181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6743e-2493-48f5-9ca5-f48da3fffe24 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:10.912223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe554158-cb18-4ba5-8a87-549c15d3ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:10.912855 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe554158-cb18-4ba5-8a87-549c15d3ab0d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:10.913129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe554158-cb18-4ba5-8a87-549c15d3ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:10.913396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe554158-cb18-4ba5-8a87-549c15d3ab0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:10.926725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe554158-cb18-4ba5-8a87-549c15d3ab0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-fe554158-cb18-4ba5-8a87-549c15d3ab0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:10.933477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:10.934588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:10.935584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 114/223] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:32:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:32:10.936284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331ec3b-7e11-495c-a45b-d5ef5dbd0097 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:10.939328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331ec3b-7e11-495c-a45b-d5ef5dbd0097 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:10.939328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331ec3b-7e11-495c-a45b-d5ef5dbd0097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:10.957804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6331ec3b-7e11-495c-a45b-d5ef5dbd0097 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:10.959205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6331ec3b-7e11-495c-a45b-d5ef5dbd0097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:11.003928 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6743e-2493-48f5-9ca5-f48da3fffe24 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:11.045824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 115/224] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:11.046903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cf7cd-8ae6-4e3d-9dc7-344aeddf70fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:11.047573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cf7cd-8ae6-4e3d-9dc7-344aeddf70fd None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:11.047851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cf7cd-8ae6-4e3d-9dc7-344aeddf70fd None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:11.048120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cf7cd-8ae6-4e3d-9dc7-344aeddf70fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:11.048844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cf7cd-8ae6-4e3d-9dc7-344aeddf70fd None admin] RBAC: Adding query filter params (role.id=efddf5f5222644b18b6836117621adf8, scope.project.id=e2499b13100b44bb8d6160b53bc3747c, user.id=e0f4af2471dd4ebf9adcf2b62a178e75) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:11.049151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cf7cd-8ae6-4e3d-9dc7-344aeddf70fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:11.050025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331ec3b-7e11-495c-a45b-d5ef5dbd0097 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:11.050642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cf7cd-8ae6-4e3d-9dc7-344aeddf70fd None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:11.060804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 110/225] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:10 2022] GET /identity/v3/role_assignments?user.id=e0f4af2471dd4ebf9adcf2b62a178e75&scope.project.id=e2499b13100b44bb8d6160b53bc3747c&role.id=efddf5f5222644b18b6836117621adf8 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:11.157400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:11.175949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331ec3b-7e11-495c-a45b-d5ef5dbd0097 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:11.233674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752c7b7-9503-429d-a509-2aeed710219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:11.234329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752c7b7-9503-429d-a509-2aeed710219f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:11.234752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752c7b7-9503-429d-a509-2aeed710219f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:11.235066 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752c7b7-9503-429d-a509-2aeed710219f None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:11.235873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752c7b7-9503-429d-a509-2aeed710219f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:11.236577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752c7b7-9503-429d-a509-2aeed710219f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:11.238785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752c7b7-9503-429d-a509-2aeed710219f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:11.250429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 116/226] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:11.256032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cf7cd-8ae6-4e3d-9dc7-344aeddf70fd None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:11.338650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752c7b7-9503-429d-a509-2aeed710219f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:11.356709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:11.410212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cf7cd-8ae6-4e3d-9dc7-344aeddf70fd None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:11.457629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1070fd-42ac-445d-8904-0065efe4bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:11.459065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1070fd-42ac-445d-8904-0065efe4bd52 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:11.459563 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1070fd-42ac-445d-8904-0065efe4bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:11.460248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1070fd-42ac-445d-8904-0065efe4bd52 None admin] PATH_INFO: `/v3/users/b28acfc573784d58bbeb74a5c8057c22` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:11.462129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1070fd-42ac-445d-8904-0065efe4bd52 None admin] RBAC: Authorizing `identity:get_user(user_id=b28acfc573784d58bbeb74a5c8057c22)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:11.463758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1070fd-42ac-445d-8904-0065efe4bd52 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:11.466681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 111/227] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:11 2022] GET /identity/v3/users/b28acfc573784d58bbeb74a5c8057c22 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:11.478987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1070fd-42ac-445d-8904-0065efe4bd52 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:11.491030 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752c7b7-9503-429d-a509-2aeed710219f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:11.557150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7316e4-3da3-4079-9776-585523b27557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:11.557855 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7316e4-3da3-4079-9776-585523b27557 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:11.558134 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7316e4-3da3-4079-9776-585523b27557 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:11.558453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7316e4-3da3-4079-9776-585523b27557 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:11.576835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7316e4-3da3-4079-9776-585523b27557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-fe7316e4-3da3-4079-9776-585523b27557 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:11.582156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:11.583017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:11.583776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 117/228] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:32:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:11.591517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7316e4-3da3-4079-9776-585523b27557 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:11.606127 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:11.626401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1070fd-42ac-445d-8904-0065efe4bd52 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:11.676100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:11.687645 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481238a8-f764-42ad-8947-1e907812bf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:11.688371 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481238a8-f764-42ad-8947-1e907812bf3b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:11.688677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481238a8-f764-42ad-8947-1e907812bf3b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:11.688949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481238a8-f764-42ad-8947-1e907812bf3b None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:11.691286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481238a8-f764-42ad-8947-1e907812bf3b None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:11.694840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481238a8-f764-42ad-8947-1e907812bf3b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:11.698602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 112/229] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:11 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:11.711212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481238a8-f764-42ad-8947-1e907812bf3b None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:11.759751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7316e4-3da3-4079-9776-585523b27557 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:11.852084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b118e0a-b14c-46c4-ab0f-b2d8fe77195f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:11.853713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b118e0a-b14c-46c4-ab0f-b2d8fe77195f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:11.854234 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b118e0a-b14c-46c4-ab0f-b2d8fe77195f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:11.854505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b118e0a-b14c-46c4-ab0f-b2d8fe77195f None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:11.855174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b118e0a-b14c-46c4-ab0f-b2d8fe77195f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:11.855477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b118e0a-b14c-46c4-ab0f-b2d8fe77195f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:11.856874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b118e0a-b14c-46c4-ab0f-b2d8fe77195f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:11.862096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481238a8-f764-42ad-8947-1e907812bf3b None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:11.878881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 118/230] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:11.885178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b118e0a-b14c-46c4-ab0f-b2d8fe77195f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:11.930463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46587ae5-15f2-4b08-ab7a-a173c5f9137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:11.931134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46587ae5-15f2-4b08-ab7a-a173c5f9137f None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:11.931408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46587ae5-15f2-4b08-ab7a-a173c5f9137f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:11.931680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46587ae5-15f2-4b08-ab7a-a173c5f9137f None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c/users/b28acfc573784d58bbeb74a5c8057c22/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:11.934643 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46587ae5-15f2-4b08-ab7a-a173c5f9137f None admin] RBAC: Authorizing `identity:create_grant(project_id=e2499b13100b44bb8d6160b53bc3747c, user_id=b28acfc573784d58bbeb74a5c8057c22, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:11.936106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46587ae5-15f2-4b08-ab7a-a173c5f9137f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:11.962668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 113/231] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:32:11 2022] PUT /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c/users/b28acfc573784d58bbeb74a5c8057c22/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:32:12.006540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:12.038068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b118e0a-b14c-46c4-ab0f-b2d8fe77195f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:12.075249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d325d73-a4d2-4a76-b75c-daee1fe3e084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:12.075889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d325d73-a4d2-4a76-b75c-daee1fe3e084 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:12.076163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d325d73-a4d2-4a76-b75c-daee1fe3e084 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:12.076430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d325d73-a4d2-4a76-b75c-daee1fe3e084 None admin] PATH_INFO: `/v3/users/b28acfc573784d58bbeb74a5c8057c22` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:12.077569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d325d73-a4d2-4a76-b75c-daee1fe3e084 None admin] RBAC: Authorizing `identity:get_user(user_id=b28acfc573784d58bbeb74a5c8057c22)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:12.079072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d325d73-a4d2-4a76-b75c-daee1fe3e084 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:12.081027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 119/232] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:11 2022] GET /identity/v3/users/b28acfc573784d58bbeb74a5c8057c22 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:12.086814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46587ae5-15f2-4b08-ab7a-a173c5f9137f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:12.152670 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:12.220525 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46587ae5-15f2-4b08-ab7a-a173c5f9137f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:12.265235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854366e7-2edd-463a-8a4b-41d0e1e9d396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:12.265954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854366e7-2edd-463a-8a4b-41d0e1e9d396 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:12.266420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854366e7-2edd-463a-8a4b-41d0e1e9d396 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:12.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854366e7-2edd-463a-8a4b-41d0e1e9d396 None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:12.270982 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854366e7-2edd-463a-8a4b-41d0e1e9d396 None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:12.273750 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854366e7-2edd-463a-8a4b-41d0e1e9d396 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:12.276704 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 114/233] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:12 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:12.282749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d325d73-a4d2-4a76-b75c-daee1fe3e084 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:12.451128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d325d73-a4d2-4a76-b75c-daee1fe3e084 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:12.490507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c882f-960e-4d3a-afd5-e19fa0afd17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:12.491175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c882f-960e-4d3a-afd5-e19fa0afd17f None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:12.492234 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c882f-960e-4d3a-afd5-e19fa0afd17f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:12.492234 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c882f-960e-4d3a-afd5-e19fa0afd17f None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c/users/b28acfc573784d58bbeb74a5c8057c22/roles/c0bf52b447564f769d707e96facb20a1` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:12.494521 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c882f-960e-4d3a-afd5-e19fa0afd17f None admin] RBAC: Authorizing `identity:create_grant(project_id=e2499b13100b44bb8d6160b53bc3747c, user_id=b28acfc573784d58bbeb74a5c8057c22, role_id=c0bf52b447564f769d707e96facb20a1)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:12.495978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c882f-960e-4d3a-afd5-e19fa0afd17f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:12.506042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:12.510879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 120/234] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:32:12 2022] PUT /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c/users/b28acfc573784d58bbeb74a5c8057c22/roles/c0bf52b447564f769d707e96facb20a1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:32:12.882153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318a8cb-787b-4f8d-a26a-851f1af9ac5f None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:12.882724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318a8cb-787b-4f8d-a26a-851f1af9ac5f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:12.882724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318a8cb-787b-4f8d-a26a-851f1af9ac5f None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:12.884108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 115/235] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:12.889446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82cbd8-889e-43fa-a943-ffe68a67f1ea None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:12.889787 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82cbd8-889e-43fa-a943-ffe68a67f1ea None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:12.890071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82cbd8-889e-43fa-a943-ffe68a67f1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:12.908832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-cd82cbd8-889e-43fa-a943-ffe68a67f1ea None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:12.913292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-cd82cbd8-889e-43fa-a943-ffe68a67f1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:12.999911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 121/236] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:13.203523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5457a-27e6-4cfb-9acf-f1da9bb44266 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:13.204143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5457a-27e6-4cfb-9acf-f1da9bb44266 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:13.204626 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5457a-27e6-4cfb-9acf-f1da9bb44266 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:13.206197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 116/237] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:13.212084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb95f3-273c-45d7-9f28-67cea8c6d216 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:13.212447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb95f3-273c-45d7-9f28-67cea8c6d216 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:13.212727 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb95f3-273c-45d7-9f28-67cea8c6d216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:13.233540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b7fb95f3-273c-45d7-9f28-67cea8c6d216 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:13.238320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b7fb95f3-273c-45d7-9f28-67cea8c6d216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:13.328093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 122/238] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:13.347360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5457a-27e6-4cfb-9acf-f1da9bb44266 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:13.509291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5457a-27e6-4cfb-9acf-f1da9bb44266 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:13.532985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:13.558515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37626c18-25b2-48cf-bbcf-963caf1a0df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:13.559165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37626c18-25b2-48cf-bbcf-963caf1a0df0 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:13.559439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37626c18-25b2-48cf-bbcf-963caf1a0df0 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:13.559717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37626c18-25b2-48cf-bbcf-963caf1a0df0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:13.560857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37626c18-25b2-48cf-bbcf-963caf1a0df0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:13.562335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37626c18-25b2-48cf-bbcf-963caf1a0df0 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:13.564197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 117/239] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:13.569588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb95f3-273c-45d7-9f28-67cea8c6d216 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:13.670439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:13.720556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb95f3-273c-45d7-9f28-67cea8c6d216 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:13.748993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37626c18-25b2-48cf-bbcf-963caf1a0df0 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:13.785170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c7514-89d6-457a-88dc-12386d8b4561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:13.785814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c7514-89d6-457a-88dc-12386d8b4561 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:13.786095 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c7514-89d6-457a-88dc-12386d8b4561 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:13.786367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c7514-89d6-457a-88dc-12386d8b4561 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:13.787026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c7514-89d6-457a-88dc-12386d8b4561 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:13.788497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c7514-89d6-457a-88dc-12386d8b4561 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:13.833793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 123/240] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:32:13 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:32:13.838212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c9ecf-3b2d-47ee-9da5-f5d41a312d4e None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:13.838499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c9ecf-3b2d-47ee-9da5-f5d41a312d4e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:13.838774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c9ecf-3b2d-47ee-9da5-f5d41a312d4e None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:13.840417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 124/241] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:32:13.850438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b17a51-d4d2-4a30-98f2-96134eb2dbfa None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:13.850720 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b17a51-d4d2-4a30-98f2-96134eb2dbfa None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:13.851089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b17a51-d4d2-4a30-98f2-96134eb2dbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:13.889566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-a0b17a51-d4d2-4a30-98f2-96134eb2dbfa None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:13.890831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a0b17a51-d4d2-4a30-98f2-96134eb2dbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:13.971045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37626c18-25b2-48cf-bbcf-963caf1a0df0 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:13.986605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 125/242] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:14.018847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e4d97-24ae-45f3-8f63-7952f01ae0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:14.019876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e4d97-24ae-45f3-8f63-7952f01ae0ed None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:14.020345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e4d97-24ae-45f3-8f63-7952f01ae0ed None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:14.020806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e4d97-24ae-45f3-8f63-7952f01ae0ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:14.032268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e4d97-24ae-45f3-8f63-7952f01ae0ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-a42e4d97-24ae-45f3-8f63-7952f01ae0ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:14.037921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:14.039291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:14.040080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:14.040080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:14.040080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:14.040080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:14.040080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:14.040080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:14.040080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:14.040080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:14.040080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:14.040080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:14.040080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:14.040080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:14.042713 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 118/243] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:14.050728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b17a51-d4d2-4a30-98f2-96134eb2dbfa None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:14.100745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:14.157838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b17a51-d4d2-4a30-98f2-96134eb2dbfa None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:14.200573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c679a75-4872-4ea4-a803-7a0d114a8149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:14.201196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c679a75-4872-4ea4-a803-7a0d114a8149 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:14.201480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c679a75-4872-4ea4-a803-7a0d114a8149 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:14.201839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c679a75-4872-4ea4-a803-7a0d114a8149 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:14.202516 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c679a75-4872-4ea4-a803-7a0d114a8149 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:14.202871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c679a75-4872-4ea4-a803-7a0d114a8149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:14.204297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c679a75-4872-4ea4-a803-7a0d114a8149 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:14.214257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 126/244] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:14.220078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e4d97-24ae-45f3-8f63-7952f01ae0ed None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:14.272254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:14.295236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c679a75-4872-4ea4-a803-7a0d114a8149 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:14.350300 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e4d97-24ae-45f3-8f63-7952f01ae0ed None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:14.393200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b954210-1467-4020-9f5b-54aecd0d01fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:14.393870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b954210-1467-4020-9f5b-54aecd0d01fc None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:14.394154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b954210-1467-4020-9f5b-54aecd0d01fc None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:14.394718 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b954210-1467-4020-9f5b-54aecd0d01fc None admin] PATH_INFO: `/v3/users/560454d9722540bda540212159c572bb` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:14.410930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b954210-1467-4020-9f5b-54aecd0d01fc None admin] RBAC: Authorizing `identity:get_user(user_id=560454d9722540bda540212159c572bb)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:14.413219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b954210-1467-4020-9f5b-54aecd0d01fc None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:14.414963 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 119/245] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:14 2022] GET /identity/v3/users/560454d9722540bda540212159c572bb => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:14.420639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b954210-1467-4020-9f5b-54aecd0d01fc None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:14.456457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c679a75-4872-4ea4-a803-7a0d114a8149 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:14.470010 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:14.507702 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126839f-2df8-44eb-81ba-0118d8a69369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:14.508603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126839f-2df8-44eb-81ba-0118d8a69369 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:14.508884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126839f-2df8-44eb-81ba-0118d8a69369 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:14.509159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126839f-2df8-44eb-81ba-0118d8a69369 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:14.522728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126839f-2df8-44eb-81ba-0118d8a69369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-8126839f-2df8-44eb-81ba-0118d8a69369 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:14.529868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:14.530852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:14.531613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:14.531613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:14.531613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:14.531613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:14.531613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:14.531613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:14.531613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:14.531613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:14.531613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:14.531613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:14.531613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:14.531613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:14.533614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 127/246] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:14.537742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126839f-2df8-44eb-81ba-0118d8a69369 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:14.554033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b954210-1467-4020-9f5b-54aecd0d01fc None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:14.556787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:14.601029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47883754-eecc-4250-b546-300a2a032bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:14.602156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47883754-eecc-4250-b546-300a2a032bff None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:14.602370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47883754-eecc-4250-b546-300a2a032bff None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:14.603385 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47883754-eecc-4250-b546-300a2a032bff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:14.675340 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126839f-2df8-44eb-81ba-0118d8a69369 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:14.696667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47883754-eecc-4250-b546-300a2a032bff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:14.699485 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47883754-eecc-4250-b546-300a2a032bff None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:14.721243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac05be-d1d9-4123-a10c-df06c8948227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:14.721977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac05be-d1d9-4123-a10c-df06c8948227 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:14.722325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac05be-d1d9-4123-a10c-df06c8948227 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:14.722662 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac05be-d1d9-4123-a10c-df06c8948227 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:14.723908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac05be-d1d9-4123-a10c-df06c8948227 None admin] RBAC: Adding query filter params (name=member) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:14.724293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac05be-d1d9-4123-a10c-df06c8948227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:14.727247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac05be-d1d9-4123-a10c-df06c8948227 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:14.739546 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 128/247] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:14.740903 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048e3f9-d42c-470c-bc99-ae858f13f711 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:14.741371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048e3f9-d42c-470c-bc99-ae858f13f711 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:14.741954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048e3f9-d42c-470c-bc99-ae858f13f711 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:14.743462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 129/248] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:14.749196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc67a8-e39b-4203-8d7b-117bfe45cd04 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:14.749814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc67a8-e39b-4203-8d7b-117bfe45cd04 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:14.750282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc67a8-e39b-4203-8d7b-117bfe45cd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:14.772026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-1cdc67a8-e39b-4203-8d7b-117bfe45cd04 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:14.774186 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1cdc67a8-e39b-4203-8d7b-117bfe45cd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:14.801098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:14.830302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 120/249] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:14.831494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47883754-eecc-4250-b546-300a2a032bff None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:14.834986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:14.857478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 130/250] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:14.859157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc67a8-e39b-4203-8d7b-117bfe45cd04 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:14.980491 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47883754-eecc-4250-b546-300a2a032bff None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:15.039028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff051b0-881b-4177-a9ac-5e5526ef13db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:15.040078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff051b0-881b-4177-a9ac-5e5526ef13db None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:15.040553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff051b0-881b-4177-a9ac-5e5526ef13db None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:15.041019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff051b0-881b-4177-a9ac-5e5526ef13db None admin] PATH_INFO: `/v3/users/e0f4af2471dd4ebf9adcf2b62a178e75` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:15.046474 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff051b0-881b-4177-a9ac-5e5526ef13db None admin] RBAC: Authorizing `identity:get_user(user_id=e0f4af2471dd4ebf9adcf2b62a178e75)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:15.048158 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff051b0-881b-4177-a9ac-5e5526ef13db None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:15.050003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 121/251] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:14 2022] GET /identity/v3/users/e0f4af2471dd4ebf9adcf2b62a178e75 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:15.056557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff051b0-881b-4177-a9ac-5e5526ef13db None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:15.102203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc67a8-e39b-4203-8d7b-117bfe45cd04 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:15.165225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:15.171542 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87299837-97be-4b11-8808-72fe4940f205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:15.172214 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87299837-97be-4b11-8808-72fe4940f205 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:15.172501 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87299837-97be-4b11-8808-72fe4940f205 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:15.172775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87299837-97be-4b11-8808-72fe4940f205 None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:15.177734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87299837-97be-4b11-8808-72fe4940f205 None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:15.181014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87299837-97be-4b11-8808-72fe4940f205 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:15.182843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 131/252] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:14 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:15.190878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87299837-97be-4b11-8808-72fe4940f205 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:15.256464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff051b0-881b-4177-a9ac-5e5526ef13db None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:15.263228 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:15.303330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5985f0b-afa8-4beb-af52-4184d5b6e5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:15.305328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5985f0b-afa8-4beb-af52-4184d5b6e5a8 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:15.306211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5985f0b-afa8-4beb-af52-4184d5b6e5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:15.307353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5985f0b-afa8-4beb-af52-4184d5b6e5a8 None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:15.309907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5985f0b-afa8-4beb-af52-4184d5b6e5a8 None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:15.312146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5985f0b-afa8-4beb-af52-4184d5b6e5a8 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:15.315581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 122/253] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:15 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:15.315716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5985f0b-afa8-4beb-af52-4184d5b6e5a8 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:15.361844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87299837-97be-4b11-8808-72fe4940f205 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:15.396930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e00ed2-9fcb-4e19-8a76-e41176758ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:15.398324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e00ed2-9fcb-4e19-8a76-e41176758ff9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:15.398765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e00ed2-9fcb-4e19-8a76-e41176758ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:15.399194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e00ed2-9fcb-4e19-8a76-e41176758ff9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:15.400054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e00ed2-9fcb-4e19-8a76-e41176758ff9 None admin] RBAC: Adding query filter params (role.id=015747495a144cb0976ad270d651b48f, scope.project.id=e2499b13100b44bb8d6160b53bc3747c, user.id=560454d9722540bda540212159c572bb) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:15.400504 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e00ed2-9fcb-4e19-8a76-e41176758ff9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:15.402192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e00ed2-9fcb-4e19-8a76-e41176758ff9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:15.408004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 132/254] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:15 2022] GET /identity/v3/role_assignments?user.id=560454d9722540bda540212159c572bb&scope.project.id=e2499b13100b44bb8d6160b53bc3747c&role.id=015747495a144cb0976ad270d651b48f => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:15.409556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e24463-f255-4cfe-bd48-95963304b338 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:15.410003 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e24463-f255-4cfe-bd48-95963304b338 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:15.410415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e24463-f255-4cfe-bd48-95963304b338 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:15.412233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 133/255] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:15.413149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e24463-f255-4cfe-bd48-95963304b338 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:15.430208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5985f0b-afa8-4beb-af52-4184d5b6e5a8 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:15.486980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472c6b2-d5d1-4370-a796-1613df7a55ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:15.487857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472c6b2-d5d1-4370-a796-1613df7a55ad None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:15.488280 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472c6b2-d5d1-4370-a796-1613df7a55ad None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:15.488686 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472c6b2-d5d1-4370-a796-1613df7a55ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:15.496827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:15.503562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472c6b2-d5d1-4370-a796-1613df7a55ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-4472c6b2-d5d1-4370-a796-1613df7a55ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:15.508736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:15.509613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:15.510446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:15.510446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:15.510446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:15.510446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:15.510446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:15.510446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:15.510446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:15.510446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:15.510446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:15.510446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:15.510446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:15.510446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:15.512931 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 123/256] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:32:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:15.514412 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72e6b3-eadf-4567-ae9b-680b521bffe3 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:15.514886 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72e6b3-eadf-4567-ae9b-680b521bffe3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:15.515313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72e6b3-eadf-4567-ae9b-680b521bffe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:15.524689 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:15.539179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-af72e6b3-eadf-4567-ae9b-680b521bffe3 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:15.540785 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-af72e6b3-eadf-4567-ae9b-680b521bffe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:15.551575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e24463-f255-4cfe-bd48-95963304b338 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:15.611181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f6f2d-4861-4745-8f22-921f6defd973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:15.612215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f6f2d-4861-4745-8f22-921f6defd973 None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:15.612635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f6f2d-4861-4745-8f22-921f6defd973 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:15.613116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f6f2d-4861-4745-8f22-921f6defd973 None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c/users/e0f4af2471dd4ebf9adcf2b62a178e75/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:15.615284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f6f2d-4861-4745-8f22-921f6defd973 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2499b13100b44bb8d6160b53bc3747c, user_id=e0f4af2471dd4ebf9adcf2b62a178e75, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:15.617143 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f6f2d-4861-4745-8f22-921f6defd973 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:15.620536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 124/257] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:15.621579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72e6b3-eadf-4567-ae9b-680b521bffe3 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:15.642353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 134/258] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:32:15 2022] PUT /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c/users/e0f4af2471dd4ebf9adcf2b62a178e75/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:32:15.792970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72e6b3-eadf-4567-ae9b-680b521bffe3 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:15.837723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:15.866719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef468aa-c6ec-4ee9-8e85-d73a7b5c3ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:15.867324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef468aa-c6ec-4ee9-8e85-d73a7b5c3ac9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:15.867796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef468aa-c6ec-4ee9-8e85-d73a7b5c3ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:15.867897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef468aa-c6ec-4ee9-8e85-d73a7b5c3ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:15.868538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef468aa-c6ec-4ee9-8e85-d73a7b5c3ac9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:15.868837 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef468aa-c6ec-4ee9-8e85-d73a7b5c3ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:15.870265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef468aa-c6ec-4ee9-8e85-d73a7b5c3ac9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:15.886969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 125/259] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:15.897436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f6f2d-4861-4745-8f22-921f6defd973 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:16.035201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f6f2d-4861-4745-8f22-921f6defd973 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:16.098952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf603cbb-b354-437d-acea-ee54ea2a7878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:16.099999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf603cbb-b354-437d-acea-ee54ea2a7878 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:16.100518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf603cbb-b354-437d-acea-ee54ea2a7878 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:16.101051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf603cbb-b354-437d-acea-ee54ea2a7878 None admin] PATH_INFO: `/v3/users/b28acfc573784d58bbeb74a5c8057c22` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:16.102446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf603cbb-b354-437d-acea-ee54ea2a7878 None admin] RBAC: Authorizing `identity:get_user(user_id=b28acfc573784d58bbeb74a5c8057c22)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:16.104352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf603cbb-b354-437d-acea-ee54ea2a7878 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:16.106174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 135/260] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:15 2022] GET /identity/v3/users/b28acfc573784d58bbeb74a5c8057c22 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:16.114769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef468aa-c6ec-4ee9-8e85-d73a7b5c3ac9 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:16.293964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef468aa-c6ec-4ee9-8e85-d73a7b5c3ac9 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:16.328823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61d18a-ca9e-4453-bc75-946325578943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:16.329665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61d18a-ca9e-4453-bc75-946325578943 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:16.330108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61d18a-ca9e-4453-bc75-946325578943 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:16.330614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61d18a-ca9e-4453-bc75-946325578943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:16.350758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf603cbb-b354-437d-acea-ee54ea2a7878 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:16.401355 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61d18a-ca9e-4453-bc75-946325578943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:16.403189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61d18a-ca9e-4453-bc75-946325578943 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:16.405998 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:16.417021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:16.509002 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 126/261] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:16.518859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61d18a-ca9e-4453-bc75-946325578943 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:16.599085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf603cbb-b354-437d-acea-ee54ea2a7878 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:16.678105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:16.686422 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7fa69-1ce8-4a7d-9ca0-84e68653ea4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:16.687303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7fa69-1ce8-4a7d-9ca0-84e68653ea4c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:16.687728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7fa69-1ce8-4a7d-9ca0-84e68653ea4c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:16.688200 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7fa69-1ce8-4a7d-9ca0-84e68653ea4c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:16.688459 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61d18a-ca9e-4453-bc75-946325578943 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:16.710859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7fa69-1ce8-4a7d-9ca0-84e68653ea4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-cba7fa69-1ce8-4a7d-9ca0-84e68653ea4c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:16.716182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:16.717032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:16.717866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 136/262] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:32:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:16.724370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7fa69-1ce8-4a7d-9ca0-84e68653ea4c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:16.727600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c8dda-dbf2-4696-80d4-d75c423926ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:16.728429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c8dda-dbf2-4696-80d4-d75c423926ba None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:16.729100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c8dda-dbf2-4696-80d4-d75c423926ba None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:16.729517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c8dda-dbf2-4696-80d4-d75c423926ba None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:16.730847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c8dda-dbf2-4696-80d4-d75c423926ba None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:16.732449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c8dda-dbf2-4696-80d4-d75c423926ba None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:16.734559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 127/263] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:16 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:16.740445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c8dda-dbf2-4696-80d4-d75c423926ba None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:16.842651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c8dda-dbf2-4696-80d4-d75c423926ba None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:16.857819 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:16.878642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91775934-991d-41b7-8a38-c05375fe0297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:16.879600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91775934-991d-41b7-8a38-c05375fe0297 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:16.880069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91775934-991d-41b7-8a38-c05375fe0297 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:16.880529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91775934-991d-41b7-8a38-c05375fe0297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:16.881433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91775934-991d-41b7-8a38-c05375fe0297 None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=e2499b13100b44bb8d6160b53bc3747c, user.id=b28acfc573784d58bbeb74a5c8057c22) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:16.881961 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91775934-991d-41b7-8a38-c05375fe0297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:16.883731 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91775934-991d-41b7-8a38-c05375fe0297 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:16.890019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7fa69-1ce8-4a7d-9ca0-84e68653ea4c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:16.890510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 128/264] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:16 2022] GET /identity/v3/role_assignments?user.id=b28acfc573784d58bbeb74a5c8057c22&scope.project.id=e2499b13100b44bb8d6160b53bc3747c&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:16.951132 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bdcba-8780-4985-8bb5-d7d6cba5c9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:16.951868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bdcba-8780-4985-8bb5-d7d6cba5c9b0 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:16.952109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bdcba-8780-4985-8bb5-d7d6cba5c9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:16.952389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bdcba-8780-4985-8bb5-d7d6cba5c9b0 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:16.953074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bdcba-8780-4985-8bb5-d7d6cba5c9b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:16.953379 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bdcba-8780-4985-8bb5-d7d6cba5c9b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:16.954824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bdcba-8780-4985-8bb5-d7d6cba5c9b0 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:16.969026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 137/265] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:16.970507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91775934-991d-41b7-8a38-c05375fe0297 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:17.014786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:17.091819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91775934-991d-41b7-8a38-c05375fe0297 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:17.133270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0b0e1-8d2f-409c-a682-c0596484c9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:17.134739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0b0e1-8d2f-409c-a682-c0596484c9a1 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:17.135212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0b0e1-8d2f-409c-a682-c0596484c9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:17.135673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0b0e1-8d2f-409c-a682-c0596484c9a1 None admin] PATH_INFO: `/v3/users/b28acfc573784d58bbeb74a5c8057c22` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:17.137096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0b0e1-8d2f-409c-a682-c0596484c9a1 None admin] RBAC: Authorizing `identity:get_user(user_id=b28acfc573784d58bbeb74a5c8057c22)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:17.138788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0b0e1-8d2f-409c-a682-c0596484c9a1 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:17.140952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 129/266] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:16 2022] GET /identity/v3/users/b28acfc573784d58bbeb74a5c8057c22 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:17.146128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bdcba-8780-4985-8bb5-d7d6cba5c9b0 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:17.253004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b9922-86d2-4da8-9c92-4a2b6935c7b9 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:17.253330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b9922-86d2-4da8-9c92-4a2b6935c7b9 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:17.253537 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b9922-86d2-4da8-9c92-4a2b6935c7b9 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:17.255160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 130/267] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:17.265740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61f925-8102-41f0-99d1-3f7e244a93f8 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:17.266034 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61f925-8102-41f0-99d1-3f7e244a93f8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:17.266311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61f925-8102-41f0-99d1-3f7e244a93f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:17.293170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bdcba-8780-4985-8bb5-d7d6cba5c9b0 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:17.298706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-1a61f925-8102-41f0-99d1-3f7e244a93f8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:17.301119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1a61f925-8102-41f0-99d1-3f7e244a93f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:17.365068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c580d-a504-43a7-acdd-e7d47f5890eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:17.367145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c580d-a504-43a7-acdd-e7d47f5890eb None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:17.368415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c580d-a504-43a7-acdd-e7d47f5890eb None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:17.369397 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c580d-a504-43a7-acdd-e7d47f5890eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:17.396410 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:17.432158 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 131/268] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:17.495519 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c580d-a504-43a7-acdd-e7d47f5890eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:17.497446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c580d-a504-43a7-acdd-e7d47f5890eb None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:17.634194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 138/269] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:17.640018 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61f925-8102-41f0-99d1-3f7e244a93f8 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:17.669444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:17.753997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c580d-a504-43a7-acdd-e7d47f5890eb None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:17.775888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61f925-8102-41f0-99d1-3f7e244a93f8 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:17.818024 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:17.833047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c05d6-9825-4434-824d-45285ab897cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:17.833704 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c05d6-9825-4434-824d-45285ab897cf None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:17.833992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c05d6-9825-4434-824d-45285ab897cf None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:17.834265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c05d6-9825-4434-824d-45285ab897cf None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:17.836046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c05d6-9825-4434-824d-45285ab897cf None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:17.837833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c05d6-9825-4434-824d-45285ab897cf None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:17.839773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 132/270] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:17 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:17.840965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2717b5-3ad4-481d-be4b-99a3c6577c02 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:17.841249 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2717b5-3ad4-481d-be4b-99a3c6577c02 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:17.841524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2717b5-3ad4-481d-be4b-99a3c6577c02 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:17.842869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 133/271] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:17.847901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2717b5-3ad4-481d-be4b-99a3c6577c02 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:17.887919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c580d-a504-43a7-acdd-e7d47f5890eb None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:17.923257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a0910-6b90-4673-ab2d-a15839ef2188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:17.923915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a0910-6b90-4673-ab2d-a15839ef2188 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:17.924202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a0910-6b90-4673-ab2d-a15839ef2188 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:17.924480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a0910-6b90-4673-ab2d-a15839ef2188 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:17.925594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a0910-6b90-4673-ab2d-a15839ef2188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:17.927220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a0910-6b90-4673-ab2d-a15839ef2188 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:17.933472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 139/272] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:17.934848 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60d54f-a3d2-48e2-be89-4488bcff2436 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:17.935137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60d54f-a3d2-48e2-be89-4488bcff2436 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:17.935416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60d54f-a3d2-48e2-be89-4488bcff2436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:17.946978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:17.956077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6f60d54f-a3d2-48e2-be89-4488bcff2436 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:17.957291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6f60d54f-a3d2-48e2-be89-4488bcff2436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:17.961920 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2717b5-3ad4-481d-be4b-99a3c6577c02 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:17.998883 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3995160-9525-4626-b812-9f5efb9e46ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:17.999495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3995160-9525-4626-b812-9f5efb9e46ad None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:17.999769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3995160-9525-4626-b812-9f5efb9e46ad None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:18.000158 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3995160-9525-4626-b812-9f5efb9e46ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:18.000888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3995160-9525-4626-b812-9f5efb9e46ad None admin] RBAC: Adding query filter params (role.id=c0bf52b447564f769d707e96facb20a1, scope.project.id=e2499b13100b44bb8d6160b53bc3747c, user.id=b28acfc573784d58bbeb74a5c8057c22) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:18.001191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3995160-9525-4626-b812-9f5efb9e46ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:18.002849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3995160-9525-4626-b812-9f5efb9e46ad None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:18.013828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 134/273] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:17 2022] GET /identity/v3/role_assignments?user.id=b28acfc573784d58bbeb74a5c8057c22&scope.project.id=e2499b13100b44bb8d6160b53bc3747c&role.id=c0bf52b447564f769d707e96facb20a1 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:18.014719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3995160-9525-4626-b812-9f5efb9e46ad None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:18.046554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 140/274] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:18.144600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3995160-9525-4626-b812-9f5efb9e46ad None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:18.182103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182b0b8-2e3f-43cf-8e79-1529a9c9d6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:18.182636 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182b0b8-2e3f-43cf-8e79-1529a9c9d6d7 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:18.182918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182b0b8-2e3f-43cf-8e79-1529a9c9d6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:18.183189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182b0b8-2e3f-43cf-8e79-1529a9c9d6d7 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:18.183851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182b0b8-2e3f-43cf-8e79-1529a9c9d6d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:18.185317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182b0b8-2e3f-43cf-8e79-1529a9c9d6d7 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:18.216529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 135/275] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:32:18 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:32:18.369741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60d54f-a3d2-48e2-be89-4488bcff2436 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:18.416614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:18.484254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60d54f-a3d2-48e2-be89-4488bcff2436 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:18.523612 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f623a-62d8-4ec2-b44b-53771ea3e0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:18.524249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f623a-62d8-4ec2-b44b-53771ea3e0f6 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:18.524527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f623a-62d8-4ec2-b44b-53771ea3e0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:18.524792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f623a-62d8-4ec2-b44b-53771ea3e0f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:18.535742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f623a-62d8-4ec2-b44b-53771ea3e0f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-7d0f623a-62d8-4ec2-b44b-53771ea3e0f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:18.541315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:18.542435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:18.543211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 141/276] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:18.550447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182b0b8-2e3f-43cf-8e79-1529a9c9d6d7 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:18.564113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:18.673067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606ef18-d274-44ab-ac68-aed99e848965 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:18.673373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606ef18-d274-44ab-ac68-aed99e848965 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:18.673612 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606ef18-d274-44ab-ac68-aed99e848965 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:18.675005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 142/277] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:18.676647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182b0b8-2e3f-43cf-8e79-1529a9c9d6d7 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:18.682642 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9381412-6deb-4b10-b5d5-eb6c8c75843d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:18.682920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9381412-6deb-4b10-b5d5-eb6c8c75843d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:18.683190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9381412-6deb-4b10-b5d5-eb6c8c75843d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:18.705190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-e9381412-6deb-4b10-b5d5-eb6c8c75843d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:18.706907 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e9381412-6deb-4b10-b5d5-eb6c8c75843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:18.716780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653114e7-5db6-4fbf-a55e-9b25e9dd4547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:18.717822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653114e7-5db6-4fbf-a55e-9b25e9dd4547 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:18.718303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653114e7-5db6-4fbf-a55e-9b25e9dd4547 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:18.718751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653114e7-5db6-4fbf-a55e-9b25e9dd4547 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:18.719595 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653114e7-5db6-4fbf-a55e-9b25e9dd4547 None admin] RBAC: Adding query filter params (name=member) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:18.720058 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653114e7-5db6-4fbf-a55e-9b25e9dd4547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:18.721794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653114e7-5db6-4fbf-a55e-9b25e9dd4547 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:18.733444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 136/278] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:18.739702 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653114e7-5db6-4fbf-a55e-9b25e9dd4547 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:18.788341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 143/279] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:18.822716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:18.856110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653114e7-5db6-4fbf-a55e-9b25e9dd4547 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:18.892291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed4e9c-7867-4a27-a18f-d8d29266933e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:18.893310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed4e9c-7867-4a27-a18f-d8d29266933e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:18.893804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed4e9c-7867-4a27-a18f-d8d29266933e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:18.894253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed4e9c-7867-4a27-a18f-d8d29266933e None admin] PATH_INFO: `/v3/users/1fc9afef36524574b7c37643223d3fda` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:18.910843 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed4e9c-7867-4a27-a18f-d8d29266933e None admin] RBAC: Authorizing `identity:get_user(user_id=1fc9afef36524574b7c37643223d3fda)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:18.912673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed4e9c-7867-4a27-a18f-d8d29266933e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:18.914828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 137/280] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:18 2022] GET /identity/v3/users/1fc9afef36524574b7c37643223d3fda => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:18.920970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9381412-6deb-4b10-b5d5-eb6c8c75843d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:18.954463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:19.031931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9381412-6deb-4b10-b5d5-eb6c8c75843d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:19.068170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc64f6-21a5-464d-852c-c14b4b1e5da0 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:19.068773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc64f6-21a5-464d-852c-c14b4b1e5da0 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:19.069218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc64f6-21a5-464d-852c-c14b4b1e5da0 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:19.070979 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 138/281] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:32:19.076033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e135b3-e115-4797-89ce-a47440eed028 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:19.076536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e135b3-e115-4797-89ce-a47440eed028 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:19.077022 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e135b3-e115-4797-89ce-a47440eed028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:19.078964 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65aaa3-41c5-40ce-b56c-affb5e3f894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:19.079587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65aaa3-41c5-40ce-b56c-affb5e3f894d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:19.079862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65aaa3-41c5-40ce-b56c-affb5e3f894d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:19.080127 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65aaa3-41c5-40ce-b56c-affb5e3f894d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:19.080855 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65aaa3-41c5-40ce-b56c-affb5e3f894d None admin] RBAC: Adding query filter params (role.id=efddf5f5222644b18b6836117621adf8, user.id=1fc9afef36524574b7c37643223d3fda, scope.system=all) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:19.081159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65aaa3-41c5-40ce-b56c-affb5e3f894d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:19.082623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65aaa3-41c5-40ce-b56c-affb5e3f894d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:19.089243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 144/282] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Oct 4 08:32:18 2022] GET /identity/v3/role_assignments?user.id=1fc9afef36524574b7c37643223d3fda&scope.system=all&role.id=efddf5f5222644b18b6836117621adf8 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:19.090086 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65aaa3-41c5-40ce-b56c-affb5e3f894d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:19.100321 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a3e135b3-e115-4797-89ce-a47440eed028 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:19.102654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a3e135b3-e115-4797-89ce-a47440eed028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:19.184786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 139/283] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:19.250269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:19.259490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65aaa3-41c5-40ce-b56c-affb5e3f894d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:19.304285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8b787-a651-41e7-9ab0-d5524b1890de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:19.304845 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8b787-a651-41e7-9ab0-d5524b1890de None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:19.305125 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8b787-a651-41e7-9ab0-d5524b1890de None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:19.305396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8b787-a651-41e7-9ab0-d5524b1890de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:19.322596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8b787-a651-41e7-9ab0-d5524b1890de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-06f8b787-a651-41e7-9ab0-d5524b1890de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:19.327473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:19.328337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:19.329520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 145/284] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:19.334686 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e135b3-e115-4797-89ce-a47440eed028 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:19.435453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:19.488165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e135b3-e115-4797-89ce-a47440eed028 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:19.510770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8b787-a651-41e7-9ab0-d5524b1890de None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:19.547788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adad237-a9a3-4589-9849-a6724e4bc6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:19.548411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adad237-a9a3-4589-9849-a6724e4bc6fe None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:19.548690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adad237-a9a3-4589-9849-a6724e4bc6fe None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:19.548960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adad237-a9a3-4589-9849-a6724e4bc6fe None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:19.549626 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adad237-a9a3-4589-9849-a6724e4bc6fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:19.549970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adad237-a9a3-4589-9849-a6724e4bc6fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:19.551353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adad237-a9a3-4589-9849-a6724e4bc6fe None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:19.567828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 140/285] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:19.574204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adad237-a9a3-4589-9849-a6724e4bc6fe None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:19.689853 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adad237-a9a3-4589-9849-a6724e4bc6fe None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:19.695094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8b787-a651-41e7-9ab0-d5524b1890de None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:19.721271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:19.733075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339570f-0c8f-4ede-bfed-99782d40507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:19.734084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339570f-0c8f-4ede-bfed-99782d40507c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:19.736087 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339570f-0c8f-4ede-bfed-99782d40507c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:19.736660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339570f-0c8f-4ede-bfed-99782d40507c None admin] PATH_INFO: `/v3/roles/member` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:19.739583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d0c56-85c5-4b81-a968-f61ab6b5887a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:19.740364 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d0c56-85c5-4b81-a968-f61ab6b5887a None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:19.740710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d0c56-85c5-4b81-a968-f61ab6b5887a None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:19.741049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d0c56-85c5-4b81-a968-f61ab6b5887a None admin] PATH_INFO: `/v3/users/560454d9722540bda540212159c572bb` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:19.742306 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d0c56-85c5-4b81-a968-f61ab6b5887a None admin] RBAC: Authorizing `identity:get_user(user_id=560454d9722540bda540212159c572bb)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:19.743900 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d0c56-85c5-4b81-a968-f61ab6b5887a None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:19.746197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 141/286] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:19 2022] GET /identity/v3/users/560454d9722540bda540212159c572bb => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:19.749545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339570f-0c8f-4ede-bfed-99782d40507c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:19.751265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d0c56-85c5-4b81-a968-f61ab6b5887a None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-0339570f-0c8f-4ede-bfed-99782d40507c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:19.754746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:19.755608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:19.756371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 146/287] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:32:19.766254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339570f-0c8f-4ede-bfed-99782d40507c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:19.799323 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:19.902971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339570f-0c8f-4ede-bfed-99782d40507c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:19.930309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d0c56-85c5-4b81-a968-f61ab6b5887a None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:19.938181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47c87a-ca74-44c7-bbc7-a9924665de9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:19.938867 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47c87a-ca74-44c7-bbc7-a9924665de9c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:19.939219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47c87a-ca74-44c7-bbc7-a9924665de9c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:19.939586 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47c87a-ca74-44c7-bbc7-a9924665de9c None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:19.940314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47c87a-ca74-44c7-bbc7-a9924665de9c None admin] RBAC: Adding query filter params (name=member) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:19.940685 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47c87a-ca74-44c7-bbc7-a9924665de9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:19.942191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47c87a-ca74-44c7-bbc7-a9924665de9c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:19.955011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 147/288] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:19.960818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47c87a-ca74-44c7-bbc7-a9924665de9c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:19.998417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c97f2-ce91-42a1-89a7-49fd2942f60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:19.999071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c97f2-ce91-42a1-89a7-49fd2942f60b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:19.999356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c97f2-ce91-42a1-89a7-49fd2942f60b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:19.999637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c97f2-ce91-42a1-89a7-49fd2942f60b None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:20.000811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c97f2-ce91-42a1-89a7-49fd2942f60b None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:20.002362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c97f2-ce91-42a1-89a7-49fd2942f60b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:20.004342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 142/289] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:19 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:20.010142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c97f2-ce91-42a1-89a7-49fd2942f60b None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:20.053241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:20.068787 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47c87a-ca74-44c7-bbc7-a9924665de9c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:20.135962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:20.141823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3b5ec-55aa-4724-9d0e-c9efd730f420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:20.142471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3b5ec-55aa-4724-9d0e-c9efd730f420 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:20.142754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3b5ec-55aa-4724-9d0e-c9efd730f420 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:20.143034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3b5ec-55aa-4724-9d0e-c9efd730f420 None admin] PATH_INFO: `/v3/users/e0f4af2471dd4ebf9adcf2b62a178e75` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:20.146399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3b5ec-55aa-4724-9d0e-c9efd730f420 None admin] RBAC: Authorizing `identity:get_user(user_id=e0f4af2471dd4ebf9adcf2b62a178e75)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:20.147922 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3b5ec-55aa-4724-9d0e-c9efd730f420 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:20.149508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 148/290] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:19 2022] GET /identity/v3/users/e0f4af2471dd4ebf9adcf2b62a178e75 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:20.156213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3b5ec-55aa-4724-9d0e-c9efd730f420 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:20.211227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c97f2-ce91-42a1-89a7-49fd2942f60b None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:20.248676 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7f976-187c-4eae-81a9-b89dd57423ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:20.249337 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7f976-187c-4eae-81a9-b89dd57423ef None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:20.249618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7f976-187c-4eae-81a9-b89dd57423ef None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:20.249929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7f976-187c-4eae-81a9-b89dd57423ef None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c/users/560454d9722540bda540212159c572bb/roles/015747495a144cb0976ad270d651b48f` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:20.251831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7f976-187c-4eae-81a9-b89dd57423ef None admin] RBAC: Authorizing `identity:create_grant(project_id=e2499b13100b44bb8d6160b53bc3747c, user_id=560454d9722540bda540212159c572bb, role_id=015747495a144cb0976ad270d651b48f)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:20.253289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7f976-187c-4eae-81a9-b89dd57423ef None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:20.272913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 143/291] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:32:20 2022] PUT /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c/users/560454d9722540bda540212159c572bb/roles/015747495a144cb0976ad270d651b48f => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:32:20.340570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3b5ec-55aa-4724-9d0e-c9efd730f420 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:20.383210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099c34e-3a2d-4404-9c87-8ad946cd95cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:20.383840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099c34e-3a2d-4404-9c87-8ad946cd95cb None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:20.384120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099c34e-3a2d-4404-9c87-8ad946cd95cb None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:20.384389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099c34e-3a2d-4404-9c87-8ad946cd95cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:20.411549 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:20.478317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099c34e-3a2d-4404-9c87-8ad946cd95cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:20.480464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099c34e-3a2d-4404-9c87-8ad946cd95cb None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:20.585651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 149/292] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:20.591468 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7f976-187c-4eae-81a9-b89dd57423ef None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:20.700418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7f976-187c-4eae-81a9-b89dd57423ef None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:20.717281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:20.735383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5d593-b67e-43b5-b669-c359a0722b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:20.736289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5d593-b67e-43b5-b669-c359a0722b0e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:20.737053 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5d593-b67e-43b5-b669-c359a0722b0e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:20.737616 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5d593-b67e-43b5-b669-c359a0722b0e None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:20.738791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5d593-b67e-43b5-b669-c359a0722b0e None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:20.740223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5d593-b67e-43b5-b669-c359a0722b0e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:20.742419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 144/293] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:20 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:20.748806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099c34e-3a2d-4404-9c87-8ad946cd95cb None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:20.805469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:20.853194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099c34e-3a2d-4404-9c87-8ad946cd95cb None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:20.888734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12507f91-611e-42dd-a898-474956123196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:20.889350 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12507f91-611e-42dd-a898-474956123196 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:20.889630 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12507f91-611e-42dd-a898-474956123196 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:20.889931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12507f91-611e-42dd-a898-474956123196 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:20.890656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12507f91-611e-42dd-a898-474956123196 None admin] RBAC: Adding query filter params (role.id=efddf5f5222644b18b6836117621adf8, scope.project.id=e2499b13100b44bb8d6160b53bc3747c, user.id=e0f4af2471dd4ebf9adcf2b62a178e75) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:20.890964 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12507f91-611e-42dd-a898-474956123196 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:20.892406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12507f91-611e-42dd-a898-474956123196 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:20.899543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 150/294] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:20 2022] GET /identity/v3/role_assignments?user.id=e0f4af2471dd4ebf9adcf2b62a178e75&scope.project.id=e2499b13100b44bb8d6160b53bc3747c&role.id=efddf5f5222644b18b6836117621adf8 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:21.155878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0313e-4ac4-46ce-87a2-dfe9465b49de None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:21.156273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0313e-4ac4-46ce-87a2-dfe9465b49de None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:21.156448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0313e-4ac4-46ce-87a2-dfe9465b49de None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:21.158169 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 145/295] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:21.164455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d4ce4-5852-49b2-aa5b-cde2fdcb7387 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:21.164898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d4ce4-5852-49b2-aa5b-cde2fdcb7387 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:21.165068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469d4ce4-5852-49b2-aa5b-cde2fdcb7387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:21.185889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-469d4ce4-5852-49b2-aa5b-cde2fdcb7387 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:21.187098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-469d4ce4-5852-49b2-aa5b-cde2fdcb7387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:21.273810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 151/296] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:21.477625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a6fc-c617-461c-8ad2-483263a36ca2 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:21.478126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a6fc-c617-461c-8ad2-483263a36ca2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:21.478126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a6fc-c617-461c-8ad2-483263a36ca2 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:21.480076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 146/297] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:21.485400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b01ae-88b9-4b03-b1f9-8924acf2c696 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:21.485723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b01ae-88b9-4b03-b1f9-8924acf2c696 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:21.486031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b01ae-88b9-4b03-b1f9-8924acf2c696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:21.511754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-c96b01ae-88b9-4b03-b1f9-8924acf2c696 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:21.512946 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c96b01ae-88b9-4b03-b1f9-8924acf2c696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:21.553725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:21.559981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a6fc-c617-461c-8ad2-483263a36ca2 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:21.605835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 152/298] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:21.678111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a6fc-c617-461c-8ad2-483263a36ca2 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:21.724601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555636fe-a028-415d-9bbc-3a32412e8631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:21.725242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555636fe-a028-415d-9bbc-3a32412e8631 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:21.725523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555636fe-a028-415d-9bbc-3a32412e8631 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:21.725847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555636fe-a028-415d-9bbc-3a32412e8631 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:21.737003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555636fe-a028-415d-9bbc-3a32412e8631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-555636fe-a028-415d-9bbc-3a32412e8631 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:21.742197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:21.743098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:21.743902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:21.743902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:21.743902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:21.743902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:21.743902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:21.743902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:21.743902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:21.743902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:21.743902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:21.743902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:21.743902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:21.743902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:21.748083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 147/299] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:32:21.749611 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b01ae-88b9-4b03-b1f9-8924acf2c696 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:21.751382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880b3e0-52a0-4a2f-a2b9-9cef9fec0d0a None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:21.751662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880b3e0-52a0-4a2f-a2b9-9cef9fec0d0a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:21.751932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880b3e0-52a0-4a2f-a2b9-9cef9fec0d0a None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:21.753738 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 148/300] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:21.764130 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b58ab-5373-4139-831b-0fe63886a184 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:21.764416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b58ab-5373-4139-831b-0fe63886a184 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:21.764689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b58ab-5373-4139-831b-0fe63886a184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:21.785842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-bd5b58ab-5373-4139-831b-0fe63886a184 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:21.787083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-bd5b58ab-5373-4139-831b-0fe63886a184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:21.838063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:21.865777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b01ae-88b9-4b03-b1f9-8924acf2c696 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:21.871182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 149/301] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:21.880696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b58ab-5373-4139-831b-0fe63886a184 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:21.909297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbc9e7-f784-474b-bf14-c200ca01b8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:21.910045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbc9e7-f784-474b-bf14-c200ca01b8ea None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:21.910396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbc9e7-f784-474b-bf14-c200ca01b8ea None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:21.910763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbc9e7-f784-474b-bf14-c200ca01b8ea None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:21.911503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbc9e7-f784-474b-bf14-c200ca01b8ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:21.911873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbc9e7-f784-474b-bf14-c200ca01b8ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:21.913359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbc9e7-f784-474b-bf14-c200ca01b8ea None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:21.927514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 153/302] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:21.933714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbc9e7-f784-474b-bf14-c200ca01b8ea None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:21.967539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:22.005271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b58ab-5373-4139-831b-0fe63886a184 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:22.042179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbc9e7-f784-474b-bf14-c200ca01b8ea None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:22.045868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff9363-2be6-4617-9639-1450f1c58a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:22.046512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff9363-2be6-4617-9639-1450f1c58a22 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:22.046795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff9363-2be6-4617-9639-1450f1c58a22 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:22.047071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff9363-2be6-4617-9639-1450f1c58a22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:22.048196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff9363-2be6-4617-9639-1450f1c58a22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:22.049796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff9363-2be6-4617-9639-1450f1c58a22 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:22.056160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 150/303] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:22.057224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff9363-2be6-4617-9639-1450f1c58a22 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:22.085147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce39fe-55d0-4d27-8630-ba21947239ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:22.086735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce39fe-55d0-4d27-8630-ba21947239ed None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:22.087242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce39fe-55d0-4d27-8630-ba21947239ed None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:22.087895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce39fe-55d0-4d27-8630-ba21947239ed None admin] PATH_INFO: `/v3/users/903b7f5cbaaa4f589a5b23eaed5e3635` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:22.107620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce39fe-55d0-4d27-8630-ba21947239ed None admin] RBAC: Authorizing `identity:get_user(user_id=903b7f5cbaaa4f589a5b23eaed5e3635)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:22.109532 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce39fe-55d0-4d27-8630-ba21947239ed None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:22.112337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 154/304] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:21 2022] GET /identity/v3/users/903b7f5cbaaa4f589a5b23eaed5e3635 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:22.114841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:22.118461 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce39fe-55d0-4d27-8630-ba21947239ed None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:22.181782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff9363-2be6-4617-9639-1450f1c58a22 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:22.218963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:22.232974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1aeb78-0bd7-4510-a609-03b5df190c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:22.233325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce39fe-55d0-4d27-8630-ba21947239ed None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:22.233591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1aeb78-0bd7-4510-a609-03b5df190c8b None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:22.233902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1aeb78-0bd7-4510-a609-03b5df190c8b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:22.235606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1aeb78-0bd7-4510-a609-03b5df190c8b None admin] PATH_INFO: `/v3/groups` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:22.236504 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1aeb78-0bd7-4510-a609-03b5df190c8b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:22.237808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1aeb78-0bd7-4510-a609-03b5df190c8b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:22.255217 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 151/305] 10.222.0.26 () {60 vars in 1134 bytes} [Tue Oct 4 08:32:22 2022] POST /identity/v3/groups => generated 229 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:32:22.255878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1aeb78-0bd7-4510-a609-03b5df190c8b None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:22.291238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a3399-85f9-4c89-8446-0378fdd39f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:22.292156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a3399-85f9-4c89-8446-0378fdd39f9f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:22.292470 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a3399-85f9-4c89-8446-0378fdd39f9f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:22.292749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a3399-85f9-4c89-8446-0378fdd39f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:22.293910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a3399-85f9-4c89-8446-0378fdd39f9f None admin] RBAC: Adding query filter params (role.id=015747495a144cb0976ad270d651b48f, user.id=903b7f5cbaaa4f589a5b23eaed5e3635, scope.system=all) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:22.293910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a3399-85f9-4c89-8446-0378fdd39f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:22.295350 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a3399-85f9-4c89-8446-0378fdd39f9f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:22.306344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 155/306] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Oct 4 08:32:22 2022] GET /identity/v3/role_assignments?user.id=903b7f5cbaaa4f589a5b23eaed5e3635&scope.system=all&role.id=015747495a144cb0976ad270d651b48f => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:22.417907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1aeb78-0bd7-4510-a609-03b5df190c8b None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:22.419864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:22.464803 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c235d3-afd4-4fc0-9a0f-e194f0253085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:22.465467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c235d3-afd4-4fc0-9a0f-e194f0253085 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:22.465777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c235d3-afd4-4fc0-9a0f-e194f0253085 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:22.466053 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c235d3-afd4-4fc0-9a0f-e194f0253085 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:22.480610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c235d3-afd4-4fc0-9a0f-e194f0253085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-06c235d3-afd4-4fc0-9a0f-e194f0253085 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:22.487279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:22.488211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:22.488974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 152/307] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:22.496028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a3399-85f9-4c89-8446-0378fdd39f9f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:22.610428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a3399-85f9-4c89-8446-0378fdd39f9f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:22.646067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4635a3c-9b6b-42b0-90f9-1f23776833ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:22.646681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4635a3c-9b6b-42b0-90f9-1f23776833ee None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:22.646963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4635a3c-9b6b-42b0-90f9-1f23776833ee None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:22.647238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4635a3c-9b6b-42b0-90f9-1f23776833ee None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:22.648476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4635a3c-9b6b-42b0-90f9-1f23776833ee None admin] RBAC: Adding query filter params (name=member) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:22.648859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4635a3c-9b6b-42b0-90f9-1f23776833ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:22.650284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4635a3c-9b6b-42b0-90f9-1f23776833ee None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:22.663846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 156/308] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:22.669741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c235d3-afd4-4fc0-9a0f-e194f0253085 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:22.788102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c235d3-afd4-4fc0-9a0f-e194f0253085 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:22.823805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789f614-788a-4252-8730-0a705376f053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:22.824503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f614-788a-4252-8730-0a705376f053 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:22.824777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f614-788a-4252-8730-0a705376f053 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:22.825047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789f614-788a-4252-8730-0a705376f053 None admin] PATH_INFO: `/v3/users/1fc9afef36524574b7c37643223d3fda` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:22.826246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789f614-788a-4252-8730-0a705376f053 None admin] RBAC: Authorizing `identity:get_user(user_id=1fc9afef36524574b7c37643223d3fda)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:22.827787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789f614-788a-4252-8730-0a705376f053 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:22.829652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 153/309] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:22 2022] GET /identity/v3/users/1fc9afef36524574b7c37643223d3fda => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:22.835331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4635a3c-9b6b-42b0-90f9-1f23776833ee None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:22.852741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:22.954400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4635a3c-9b6b-42b0-90f9-1f23776833ee None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:22.996731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb83bfc-d1ed-46d6-ac10-693ed2e31690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:22.997394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb83bfc-d1ed-46d6-ac10-693ed2e31690 None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:22.997678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb83bfc-d1ed-46d6-ac10-693ed2e31690 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:22.997997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb83bfc-d1ed-46d6-ac10-693ed2e31690 None admin] PATH_INFO: `/v3/system/users/1fc9afef36524574b7c37643223d3fda/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:22.999714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb83bfc-d1ed-46d6-ac10-693ed2e31690 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1fc9afef36524574b7c37643223d3fda, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:23.001246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb83bfc-d1ed-46d6-ac10-693ed2e31690 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:23.013870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 157/310] 10.222.0.26 () {58 vars in 1334 bytes} [Tue Oct 4 08:32:22 2022] PUT /identity/v3/system/users/1fc9afef36524574b7c37643223d3fda/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:32:23.029896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198951a-2fd6-447c-9eff-4729e1c7c019 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:23.030194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198951a-2fd6-447c-9eff-4729e1c7c019 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:23.030465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198951a-2fd6-447c-9eff-4729e1c7c019 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:23.032160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 154/311] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:23.038012 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f615bae-ef09-4f58-b919-c1f7ee1f432d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:23.038348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f615bae-ef09-4f58-b919-c1f7ee1f432d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:23.039169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f615bae-ef09-4f58-b919-c1f7ee1f432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:23.058820 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6f615bae-ef09-4f58-b919-c1f7ee1f432d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:23.060061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6f615bae-ef09-4f58-b919-c1f7ee1f432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:23.141754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 158/312] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:23.415076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198951a-2fd6-447c-9eff-4729e1c7c019 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:23.446215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:23.523404 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198951a-2fd6-447c-9eff-4729e1c7c019 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:23.557013 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c832e20-5e95-4279-9677-05134910fea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:23.557651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c832e20-5e95-4279-9677-05134910fea3 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:23.557956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c832e20-5e95-4279-9677-05134910fea3 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:23.558230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c832e20-5e95-4279-9677-05134910fea3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:23.569143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c832e20-5e95-4279-9677-05134910fea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-3c832e20-5e95-4279-9677-05134910fea3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:23.574049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:23.574912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:23.575680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 155/313] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:23.582273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f615bae-ef09-4f58-b919-c1f7ee1f432d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:23.594012 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:23.700130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f615bae-ef09-4f58-b919-c1f7ee1f432d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:23.737884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2902c9-713f-4b05-98b6-86d32061f367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:23.738502 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2902c9-713f-4b05-98b6-86d32061f367 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:23.738782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2902c9-713f-4b05-98b6-86d32061f367 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:23.739054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2902c9-713f-4b05-98b6-86d32061f367 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:23.739732 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2902c9-713f-4b05-98b6-86d32061f367 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:23.740036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2902c9-713f-4b05-98b6-86d32061f367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:23.742035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2902c9-713f-4b05-98b6-86d32061f367 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:23.751387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 159/314] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:23.757177 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c832e20-5e95-4279-9677-05134910fea3 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:23.853076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:23.863413 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c832e20-5e95-4279-9677-05134910fea3 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:23.898378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5c242-7bcc-41a4-8c0d-29de372c06d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:23.899008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5c242-7bcc-41a4-8c0d-29de372c06d9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:23.899289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5c242-7bcc-41a4-8c0d-29de372c06d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:23.899562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5c242-7bcc-41a4-8c0d-29de372c06d9 None admin] PATH_INFO: `/v3/users/560454d9722540bda540212159c572bb` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:23.900704 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5c242-7bcc-41a4-8c0d-29de372c06d9 None admin] RBAC: Authorizing `identity:get_user(user_id=560454d9722540bda540212159c572bb)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:23.902184 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5c242-7bcc-41a4-8c0d-29de372c06d9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:23.904205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 156/315] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:23 2022] GET /identity/v3/users/560454d9722540bda540212159c572bb => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:23.909638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2902c9-713f-4b05-98b6-86d32061f367 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:23.986467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:24.019524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2902c9-713f-4b05-98b6-86d32061f367 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:24.054201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4f70b-056a-4b42-8edd-78efa6c56de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:24.054942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4f70b-056a-4b42-8edd-78efa6c56de2 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:24.055229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4f70b-056a-4b42-8edd-78efa6c56de2 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:24.055511 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4f70b-056a-4b42-8edd-78efa6c56de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:24.127051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4f70b-056a-4b42-8edd-78efa6c56de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:24.128533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4f70b-056a-4b42-8edd-78efa6c56de2 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:24.225210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:24.262128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 160/316] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:24.266874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5c242-7bcc-41a4-8c0d-29de372c06d9 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:24.383594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5c242-7bcc-41a4-8c0d-29de372c06d9 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:24.421394 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f51c46-5adc-43a1-9a78-17f989f4c5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:24.422057 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f51c46-5adc-43a1-9a78-17f989f4c5f6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:24.422340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f51c46-5adc-43a1-9a78-17f989f4c5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:24.422614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f51c46-5adc-43a1-9a78-17f989f4c5f6 None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:24.423743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f51c46-5adc-43a1-9a78-17f989f4c5f6 None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:24.425215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f51c46-5adc-43a1-9a78-17f989f4c5f6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:24.427203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 157/317] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:24 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:24.435071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4f70b-056a-4b42-8edd-78efa6c56de2 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:24.546031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4f70b-056a-4b42-8edd-78efa6c56de2 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:24.588644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7b94e-a813-4835-94db-07b69cc9d93f None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:24.589464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7b94e-a813-4835-94db-07b69cc9d93f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:24.589464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7b94e-a813-4835-94db-07b69cc9d93f None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:24.590596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c38fe8-33d1-4e40-b046-b3bd7606c577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:24.591215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c38fe8-33d1-4e40-b046-b3bd7606c577 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:24.591499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c38fe8-33d1-4e40-b046-b3bd7606c577 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:24.591774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c38fe8-33d1-4e40-b046-b3bd7606c577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:24.592005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 158/318] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:32:24.592527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c38fe8-33d1-4e40-b046-b3bd7606c577 None admin] RBAC: Adding query filter params (role.id=015747495a144cb0976ad270d651b48f, scope.project.id=e2499b13100b44bb8d6160b53bc3747c, user.id=560454d9722540bda540212159c572bb) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:24.592839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c38fe8-33d1-4e40-b046-b3bd7606c577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:24.594347 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c38fe8-33d1-4e40-b046-b3bd7606c577 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:24.596668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402d010-3962-4886-a6c7-8ffe1090c46e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:24.597062 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402d010-3962-4886-a6c7-8ffe1090c46e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:24.597343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402d010-3962-4886-a6c7-8ffe1090c46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:24.602329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 161/319] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:24 2022] GET /identity/v3/role_assignments?user.id=560454d9722540bda540212159c572bb&scope.project.id=e2499b13100b44bb8d6160b53bc3747c&role.id=015747495a144cb0976ad270d651b48f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:24.619106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d402d010-3962-4886-a6c7-8ffe1090c46e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:24.620353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d402d010-3962-4886-a6c7-8ffe1090c46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:24.644428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:24.703051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 159/320] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:24.706412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0896d0-07bf-4a7f-853e-5f49ab9fa86d None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:24.706966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0896d0-07bf-4a7f-853e-5f49ab9fa86d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:24.707449 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0896d0-07bf-4a7f-853e-5f49ab9fa86d None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:24.709349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 162/321] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:24.715929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8773dea-695a-4ab0-a6d6-3ac94aad3069 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:24.716210 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8773dea-695a-4ab0-a6d6-3ac94aad3069 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:24.716484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8773dea-695a-4ab0-a6d6-3ac94aad3069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:24.736125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c8773dea-695a-4ab0-a6d6-3ac94aad3069 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:24.737581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c8773dea-695a-4ab0-a6d6-3ac94aad3069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:24.814185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 160/322] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:24.975509 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0896d0-07bf-4a7f-853e-5f49ab9fa86d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:25.013198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:25.082424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0896d0-07bf-4a7f-853e-5f49ab9fa86d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:25.092291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8773dea-695a-4ab0-a6d6-3ac94aad3069 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:25.119382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22772971-5b9c-4afe-a0f0-d0f21d3e6315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:25.120131 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22772971-5b9c-4afe-a0f0-d0f21d3e6315 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:25.120409 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22772971-5b9c-4afe-a0f0-d0f21d3e6315 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:25.120682 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22772971-5b9c-4afe-a0f0-d0f21d3e6315 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:25.121964 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22772971-5b9c-4afe-a0f0-d0f21d3e6315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:25.123575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22772971-5b9c-4afe-a0f0-d0f21d3e6315 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:25.125403 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 163/323] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:25.130741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22772971-5b9c-4afe-a0f0-d0f21d3e6315 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:25.183391 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:25.202846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8773dea-695a-4ab0-a6d6-3ac94aad3069 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:25.249296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b05614-e1be-4194-ba12-99e21beb8a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:25.249965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b05614-e1be-4194-ba12-99e21beb8a86 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:25.250248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b05614-e1be-4194-ba12-99e21beb8a86 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:25.250522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b05614-e1be-4194-ba12-99e21beb8a86 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:25.258092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22772971-5b9c-4afe-a0f0-d0f21d3e6315 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:25.259816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:25.262931 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b05614-e1be-4194-ba12-99e21beb8a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-99b05614-e1be-4194-ba12-99e21beb8a86 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:25.269646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:25.270567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:25.271372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 161/324] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:25.277257 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b05614-e1be-4194-ba12-99e21beb8a86 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:25.298613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c7e9f-7124-46fb-b087-fa03557b20f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:25.299431 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c7e9f-7124-46fb-b087-fa03557b20f9 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:25.299675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c7e9f-7124-46fb-b087-fa03557b20f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:25.299952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c7e9f-7124-46fb-b087-fa03557b20f9 None admin] PATH_INFO: `/v3/groups` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:25.301221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c7e9f-7124-46fb-b087-fa03557b20f9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:25.302041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c7e9f-7124-46fb-b087-fa03557b20f9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:25.321740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 164/325] 10.222.0.26 () {60 vars in 1134 bytes} [Tue Oct 4 08:32:25 2022] POST /identity/v3/groups => generated 226 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:32:25.321740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18591797-c5d1-4638-9daa-ba504e90775b None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:25.322402 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18591797-c5d1-4638-9daa-ba504e90775b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:25.322694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18591797-c5d1-4638-9daa-ba504e90775b None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:25.323994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 165/326] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:25.336433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad9704-56d8-483d-8e9c-63c0a64d81be None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:25.336612 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad9704-56d8-483d-8e9c-63c0a64d81be None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:25.336893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad9704-56d8-483d-8e9c-63c0a64d81be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:25.358476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-ccad9704-56d8-483d-8e9c-63c0a64d81be None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:25.360191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ccad9704-56d8-483d-8e9c-63c0a64d81be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:25.411978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b05614-e1be-4194-ba12-99e21beb8a86 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:25.443910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 166/327] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:25.457068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeda0dd-8571-4784-a6e0-f28a9450f720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:25.457732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeda0dd-8571-4784-a6e0-f28a9450f720 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:25.458017 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeda0dd-8571-4784-a6e0-f28a9450f720 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:25.458288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeda0dd-8571-4784-a6e0-f28a9450f720 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:25.458977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeda0dd-8571-4784-a6e0-f28a9450f720 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:25.459284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeda0dd-8571-4784-a6e0-f28a9450f720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:25.460785 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeda0dd-8571-4784-a6e0-f28a9450f720 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:25.474098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 162/328] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:25.480657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad9704-56d8-483d-8e9c-63c0a64d81be None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:25.577831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:25.589021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad9704-56d8-483d-8e9c-63c0a64d81be None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:25.628050 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a53f5a-a200-4650-8c94-cf69fec22564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:25.628765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a53f5a-a200-4650-8c94-cf69fec22564 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:25.629046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a53f5a-a200-4650-8c94-cf69fec22564 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:25.629322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a53f5a-a200-4650-8c94-cf69fec22564 None admin] PATH_INFO: `/v3/users/903b7f5cbaaa4f589a5b23eaed5e3635` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:25.630549 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a53f5a-a200-4650-8c94-cf69fec22564 None admin] RBAC: Authorizing `identity:get_user(user_id=903b7f5cbaaa4f589a5b23eaed5e3635)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:25.632033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a53f5a-a200-4650-8c94-cf69fec22564 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:25.634010 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 167/329] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:25 2022] GET /identity/v3/users/903b7f5cbaaa4f589a5b23eaed5e3635 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:25.639345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeda0dd-8571-4784-a6e0-f28a9450f720 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:25.652641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:25.729723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a53f5a-a200-4650-8c94-cf69fec22564 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:25.774153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeda0dd-8571-4784-a6e0-f28a9450f720 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:25.819032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d280ed7-bbe0-4a84-b553-2e0f173b0f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:25.820047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d280ed7-bbe0-4a84-b553-2e0f173b0f1d None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:25.820498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d280ed7-bbe0-4a84-b553-2e0f173b0f1d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:25.820919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d280ed7-bbe0-4a84-b553-2e0f173b0f1d None admin] PATH_INFO: `/v3/system/users/903b7f5cbaaa4f589a5b23eaed5e3635/roles/015747495a144cb0976ad270d651b48f` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:25.823112 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d280ed7-bbe0-4a84-b553-2e0f173b0f1d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=903b7f5cbaaa4f589a5b23eaed5e3635, role_id=015747495a144cb0976ad270d651b48f)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:25.824561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d280ed7-bbe0-4a84-b553-2e0f173b0f1d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:25.840101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 163/330] 10.222.0.26 () {58 vars in 1334 bytes} [Tue Oct 4 08:32:25 2022] PUT /identity/v3/system/users/903b7f5cbaaa4f589a5b23eaed5e3635/roles/015747495a144cb0976ad270d651b48f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:32:25.866770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a53f5a-a200-4650-8c94-cf69fec22564 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:25.908642 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0d092-4256-42e1-ae4b-e74bb192d8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:25.909308 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0d092-4256-42e1-ae4b-e74bb192d8be None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:25.909593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0d092-4256-42e1-ae4b-e74bb192d8be None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:25.909897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb0d092-4256-42e1-ae4b-e74bb192d8be None admin] PATH_INFO: `/v3/roles/member` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:25.924606 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb0d092-4256-42e1-ae4b-e74bb192d8be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-dbb0d092-4256-42e1-ae4b-e74bb192d8be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:25.931774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:25.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:25.933996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 168/331] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:32:25.942869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d280ed7-bbe0-4a84-b553-2e0f173b0f1d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:26.024507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:26.063491 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d280ed7-bbe0-4a84-b553-2e0f173b0f1d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:26.100460 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c8612-68a1-490e-8027-96102344cf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:26.101334 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c8612-68a1-490e-8027-96102344cf07 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:26.101617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c8612-68a1-490e-8027-96102344cf07 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:26.101921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c8612-68a1-490e-8027-96102344cf07 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:26.102595 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c8612-68a1-490e-8027-96102344cf07 None admin] RBAC: Adding query filter params (name=member) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:26.102897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c8612-68a1-490e-8027-96102344cf07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:26.104554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c8612-68a1-490e-8027-96102344cf07 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:26.115088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 164/332] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:26.120908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0d092-4256-42e1-ae4b-e74bb192d8be None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:26.147259 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:26.246226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb0d092-4256-42e1-ae4b-e74bb192d8be None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:26.293854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbd1bd-e669-4997-adfa-fe41374da950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:26.294567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd1bd-e669-4997-adfa-fe41374da950 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:26.294850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd1bd-e669-4997-adfa-fe41374da950 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:26.295125 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbd1bd-e669-4997-adfa-fe41374da950 None admin] PATH_INFO: `/v3/users/1fc9afef36524574b7c37643223d3fda` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:26.296589 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbd1bd-e669-4997-adfa-fe41374da950 None admin] RBAC: Authorizing `identity:get_user(user_id=1fc9afef36524574b7c37643223d3fda)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:26.298090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbd1bd-e669-4997-adfa-fe41374da950 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:26.300070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 169/333] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:26 2022] GET /identity/v3/users/1fc9afef36524574b7c37643223d3fda => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:26.306458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c8612-68a1-490e-8027-96102344cf07 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:26.431583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c8612-68a1-490e-8027-96102344cf07 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:26.480556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8177470-a85b-4f46-ade5-dd70edd968c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:26.481173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8177470-a85b-4f46-ade5-dd70edd968c6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:26.481476 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8177470-a85b-4f46-ade5-dd70edd968c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:26.481776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8177470-a85b-4f46-ade5-dd70edd968c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:26.482516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8177470-a85b-4f46-ade5-dd70edd968c6 None admin] RBAC: Adding query filter params (role.id=efddf5f5222644b18b6836117621adf8, user.id=1fc9afef36524574b7c37643223d3fda, scope.system=all) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:26.482881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8177470-a85b-4f46-ade5-dd70edd968c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:26.484387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8177470-a85b-4f46-ade5-dd70edd968c6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:26.486145 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:26.499159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 165/334] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Oct 4 08:32:26 2022] GET /identity/v3/role_assignments?user.id=1fc9afef36524574b7c37643223d3fda&scope.system=all&role.id=efddf5f5222644b18b6836117621adf8 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:27.827143 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ea6a2-b74c-4374-b020-84d6276175a0 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:27.827644 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ea6a2-b74c-4374-b020-84d6276175a0 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:27.827749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ea6a2-b74c-4374-b020-84d6276175a0 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:27.829141 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 170/335] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:27.834893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692bfd8-7700-40ab-b1b2-9dfb21930bf3 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:27.835531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692bfd8-7700-40ab-b1b2-9dfb21930bf3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:27.835719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692bfd8-7700-40ab-b1b2-9dfb21930bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:27.870552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f692bfd8-7700-40ab-b1b2-9dfb21930bf3 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:27.871764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f692bfd8-7700-40ab-b1b2-9dfb21930bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:27.961521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 166/336] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:28.482993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4a4cf-38d4-4400-83fa-11fa0aaf927e None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:28.483357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4a4cf-38d4-4400-83fa-11fa0aaf927e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:28.483552 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4a4cf-38d4-4400-83fa-11fa0aaf927e None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:28.485027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 171/337] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:28.490690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe320cc-f34b-4677-bba8-d9d1ad2a9b7f None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:28.490979 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe320cc-f34b-4677-bba8-d9d1ad2a9b7f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:28.491254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe320cc-f34b-4677-bba8-d9d1ad2a9b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:28.519491 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-7fe320cc-f34b-4677-bba8-d9d1ad2a9b7f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:28.521554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7fe320cc-f34b-4677-bba8-d9d1ad2a9b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:28.534692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc1737-727f-4ca3-99b6-191a061c07ef None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:28.534989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc1737-727f-4ca3-99b6-191a061c07ef None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:28.535244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc1737-727f-4ca3-99b6-191a061c07ef None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:28.536722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 172/338] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:32:28.543378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8531c-78d4-4cc4-9e51-7b8837ba76b2 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:28.543378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8531c-78d4-4cc4-9e51-7b8837ba76b2 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:28.543378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8531c-78d4-4cc4-9e51-7b8837ba76b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:28.570405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-9eb8531c-78d4-4cc4-9e51-7b8837ba76b2 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:28.573575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9eb8531c-78d4-4cc4-9e51-7b8837ba76b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:28.675000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 167/339] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:28.676097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe320cc-f34b-4677-bba8-d9d1ad2a9b7f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:28.681313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 173/340] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:28.758707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:28.814012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe320cc-f34b-4677-bba8-d9d1ad2a9b7f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:28.859631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce979b-6509-40ae-b2dd-3234d7557394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:28.860356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce979b-6509-40ae-b2dd-3234d7557394 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:28.860539 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce979b-6509-40ae-b2dd-3234d7557394 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:28.860810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce979b-6509-40ae-b2dd-3234d7557394 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:28.878504 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ce979b-6509-40ae-b2dd-3234d7557394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-22ce979b-6509-40ae-b2dd-3234d7557394 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:28.883435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:28.884314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:28.885103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 168/341] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:28.890636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8531c-78d4-4cc4-9e51-7b8837ba76b2 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:28.934934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:29.068533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8531c-78d4-4cc4-9e51-7b8837ba76b2 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:29.133004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02317d9e-a60a-45eb-b1c1-2bbda4134af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:29.134065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02317d9e-a60a-45eb-b1c1-2bbda4134af4 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:29.134560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02317d9e-a60a-45eb-b1c1-2bbda4134af4 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:29.134668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce979b-6509-40ae-b2dd-3234d7557394 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:29.135286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02317d9e-a60a-45eb-b1c1-2bbda4134af4 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:29.136307 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02317d9e-a60a-45eb-b1c1-2bbda4134af4 None admin] RBAC: Adding query filter params (name=member) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:29.136830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02317d9e-a60a-45eb-b1c1-2bbda4134af4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:29.138734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02317d9e-a60a-45eb-b1c1-2bbda4134af4 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:29.154663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 174/342] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:29.159659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02317d9e-a60a-45eb-b1c1-2bbda4134af4 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:29.290372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:29.311205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce979b-6509-40ae-b2dd-3234d7557394 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:29.312402 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02317d9e-a60a-45eb-b1c1-2bbda4134af4 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:29.327701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:29.351637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3d8e0-6764-4308-8e57-836745fe0a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:29.352265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3d8e0-6764-4308-8e57-836745fe0a26 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:29.352540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3d8e0-6764-4308-8e57-836745fe0a26 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:29.352786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654c338-47b1-49d7-8d3e-4ce47446c54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:29.352901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3d8e0-6764-4308-8e57-836745fe0a26 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:29.353725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654c338-47b1-49d7-8d3e-4ce47446c54d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:29.354454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654c338-47b1-49d7-8d3e-4ce47446c54d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:29.355009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654c338-47b1-49d7-8d3e-4ce47446c54d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:29.364529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3d8e0-6764-4308-8e57-836745fe0a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-b3c3d8e0-6764-4308-8e57-836745fe0a26 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:29.371250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:29.372247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:29.373051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 169/343] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:29.373747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3d8e0-6764-4308-8e57-836745fe0a26 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:29.441512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654c338-47b1-49d7-8d3e-4ce47446c54d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:29.443584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654c338-47b1-49d7-8d3e-4ce47446c54d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:29.485069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3d8e0-6764-4308-8e57-836745fe0a26 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:29.519227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2a10f-dffb-4db0-831d-aae07ba3f600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:29.519865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2a10f-dffb-4db0-831d-aae07ba3f600 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:29.520144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2a10f-dffb-4db0-831d-aae07ba3f600 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:29.520416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2a10f-dffb-4db0-831d-aae07ba3f600 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:29.531519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2a10f-dffb-4db0-831d-aae07ba3f600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-71e2a10f-dffb-4db0-831d-aae07ba3f600 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:29.538700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:29.539623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:29.540503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:29.540503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:29.540503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:29.540503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:29.540503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:29.540503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:29.540503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:29.540503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:29.540503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:29.540503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:29.540503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:29.540503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:29.541104 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:29.541182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 170/344] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:32:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:29.546313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba5d83-0210-45e4-9d76-6b358eaad512 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:29.546638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba5d83-0210-45e4-9d76-6b358eaad512 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:29.546921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba5d83-0210-45e4-9d76-6b358eaad512 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:29.548536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 171/345] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:29.548935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba5d83-0210-45e4-9d76-6b358eaad512 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:29.568230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 175/346] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 08:32:29.570074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654c338-47b1-49d7-8d3e-4ce47446c54d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:29.583164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:29.667123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba5d83-0210-45e4-9d76-6b358eaad512 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:29.701854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654c338-47b1-49d7-8d3e-4ce47446c54d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:29.703573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05f2a5-0ab4-4544-95c9-259093efad7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:29.704186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05f2a5-0ab4-4544-95c9-259093efad7b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:29.704466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05f2a5-0ab4-4544-95c9-259093efad7b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:29.704737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05f2a5-0ab4-4544-95c9-259093efad7b None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:29.705402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05f2a5-0ab4-4544-95c9-259093efad7b None admin] RBAC: Adding query filter params (name=reader) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:29.706446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05f2a5-0ab4-4544-95c9-259093efad7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:29.707094 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05f2a5-0ab4-4544-95c9-259093efad7b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:29.716828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 172/347] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:29.718078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c4403-0bd7-4202-8cab-0f65832d1a90 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:29.718356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c4403-0bd7-4202-8cab-0f65832d1a90 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:29.718769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c4403-0bd7-4202-8cab-0f65832d1a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:29.738383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-2a8c4403-0bd7-4202-8cab-0f65832d1a90 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:29.739562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2a8c4403-0bd7-4202-8cab-0f65832d1a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:29.743815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663b71d-ed3b-487f-8540-88ab9fc8ce71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:29.744689 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663b71d-ed3b-487f-8540-88ab9fc8ce71 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:29.745172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663b71d-ed3b-487f-8540-88ab9fc8ce71 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:29.745645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663b71d-ed3b-487f-8540-88ab9fc8ce71 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:29.746618 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663b71d-ed3b-487f-8540-88ab9fc8ce71 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:29.747127 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663b71d-ed3b-487f-8540-88ab9fc8ce71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:29.748944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663b71d-ed3b-487f-8540-88ab9fc8ce71 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:29.758555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 176/348] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:29.759735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663b71d-ed3b-487f-8540-88ab9fc8ce71 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:29.813022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:29.814216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 173/349] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:29.815108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c4403-0bd7-4202-8cab-0f65832d1a90 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:29.843217 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:29.908626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663b71d-ed3b-487f-8540-88ab9fc8ce71 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:29.925433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c4403-0bd7-4202-8cab-0f65832d1a90 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:29.962281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471df8b-d0cd-43b2-9809-e7e41df7c5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:29.963257 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471df8b-d0cd-43b2-9809-e7e41df7c5c5 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:29.963538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471df8b-d0cd-43b2-9809-e7e41df7c5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:29.963819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471df8b-d0cd-43b2-9809-e7e41df7c5c5 None admin] PATH_INFO: `/v3/users/903b7f5cbaaa4f589a5b23eaed5e3635` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:29.964851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d7d23-c023-47c6-b1c6-bccf19ca04b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:29.965043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471df8b-d0cd-43b2-9809-e7e41df7c5c5 None admin] RBAC: Authorizing `identity:get_user(user_id=903b7f5cbaaa4f589a5b23eaed5e3635)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:29.965912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d7d23-c023-47c6-b1c6-bccf19ca04b7 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:29.966426 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d7d23-c023-47c6-b1c6-bccf19ca04b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:29.966527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471df8b-d0cd-43b2-9809-e7e41df7c5c5 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:29.966982 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d7d23-c023-47c6-b1c6-bccf19ca04b7 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:29.968419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 174/350] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:32:29 2022] GET /identity/v3/users/903b7f5cbaaa4f589a5b23eaed5e3635 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:29.969206 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471df8b-d0cd-43b2-9809-e7e41df7c5c5 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:29.969503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d7d23-c023-47c6-b1c6-bccf19ca04b7 None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:29.971253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d7d23-c023-47c6-b1c6-bccf19ca04b7 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:29.973866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 177/351] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:29 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:29.974909 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d7d23-c023-47c6-b1c6-bccf19ca04b7 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:30.071719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:30.078270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471df8b-d0cd-43b2-9809-e7e41df7c5c5 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:30.118625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8e35b-37a3-4457-ada6-a6b5ac222238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:30.119570 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8e35b-37a3-4457-ada6-a6b5ac222238 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:30.119897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8e35b-37a3-4457-ada6-a6b5ac222238 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:30.120173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8e35b-37a3-4457-ada6-a6b5ac222238 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:30.137629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d7d23-c023-47c6-b1c6-bccf19ca04b7 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:30.158116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:30.186047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a4030-6bf9-472a-8710-c4e2e08ca998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:30.186658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a4030-6bf9-472a-8710-c4e2e08ca998 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:30.187032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a4030-6bf9-472a-8710-c4e2e08ca998 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:30.187325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a4030-6bf9-472a-8710-c4e2e08ca998 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:30.188149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a4030-6bf9-472a-8710-c4e2e08ca998 None admin] RBAC: Adding query filter params (role.id=015747495a144cb0976ad270d651b48f, user.id=903b7f5cbaaa4f589a5b23eaed5e3635, scope.system=all) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:30.188462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a4030-6bf9-472a-8710-c4e2e08ca998 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:30.191164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a4030-6bf9-472a-8710-c4e2e08ca998 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:30.204856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8e35b-37a3-4457-ada6-a6b5ac222238 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:30.205423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 178/352] 10.222.0.26 () {56 vars in 1308 bytes} [Tue Oct 4 08:32:29 2022] GET /identity/v3/role_assignments?user.id=903b7f5cbaaa4f589a5b23eaed5e3635&scope.system=all&role.id=015747495a144cb0976ad270d651b48f => generated 510 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:30.206309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a4030-6bf9-472a-8710-c4e2e08ca998 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:30.206424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8e35b-37a3-4457-ada6-a6b5ac222238 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:30.317439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:30.329970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 175/353] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:30.331487 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5b18e-67eb-4e18-a8cd-980aadacb707 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:30.332190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5b18e-67eb-4e18-a8cd-980aadacb707 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:30.332606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5b18e-67eb-4e18-a8cd-980aadacb707 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:30.334277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 176/354] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:30.342092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05896e-f482-4a24-95c0-3f4e8b358dd6 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:30.342564 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05896e-f482-4a24-95c0-3f4e8b358dd6 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:30.343003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05896e-f482-4a24-95c0-3f4e8b358dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:30.363011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a4030-6bf9-472a-8710-c4e2e08ca998 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:30.373503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-3f05896e-f482-4a24-95c0-3f4e8b358dd6 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:30.374653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3f05896e-f482-4a24-95c0-3f4e8b358dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:30.426326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136eb5d-6260-4b23-be88-81bdeb178fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:30.426978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136eb5d-6260-4b23-be88-81bdeb178fff None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:30.427254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136eb5d-6260-4b23-be88-81bdeb178fff None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:30.427526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136eb5d-6260-4b23-be88-81bdeb178fff None admin] PATH_INFO: `/v3/groups/b3128d1154244e50a747a074c4abde20` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:30.439101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136eb5d-6260-4b23-be88-81bdeb178fff None admin] RBAC: Authorizing `identity:get_group(group_id=b3128d1154244e50a747a074c4abde20)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:30.440663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136eb5d-6260-4b23-be88-81bdeb178fff None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:30.444285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 179/355] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:30 2022] GET /identity/v3/groups/b3128d1154244e50a747a074c4abde20 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:30.445839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136eb5d-6260-4b23-be88-81bdeb178fff None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:30.489756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 177/356] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:30.490675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:30.493115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05896e-f482-4a24-95c0-3f4e8b358dd6 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:30.595749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136eb5d-6260-4b23-be88-81bdeb178fff None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:30.643288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380ce1f-12e2-4319-8e5c-90e0183a1525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:30.643842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380ce1f-12e2-4319-8e5c-90e0183a1525 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:30.644120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380ce1f-12e2-4319-8e5c-90e0183a1525 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:30.644390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380ce1f-12e2-4319-8e5c-90e0183a1525 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:30.645513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380ce1f-12e2-4319-8e5c-90e0183a1525 None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:30.645844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05896e-f482-4a24-95c0-3f4e8b358dd6 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:30.646974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380ce1f-12e2-4319-8e5c-90e0183a1525 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:30.651658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 180/357] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:30 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:30.651658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380ce1f-12e2-4319-8e5c-90e0183a1525 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:30.688143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:30.708137 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cdeef-c2b6-479a-a027-7ea80e51f472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:30.709602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cdeef-c2b6-479a-a027-7ea80e51f472 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:30.710179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cdeef-c2b6-479a-a027-7ea80e51f472 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:30.711016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cdeef-c2b6-479a-a027-7ea80e51f472 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:30.726316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cdeef-c2b6-479a-a027-7ea80e51f472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-997cdeef-c2b6-479a-a027-7ea80e51f472 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:30.733565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:30.734475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:30.735244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:30.735244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:30.735244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:30.735244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:30.735244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:30.735244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:30.735244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:30.735244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:30.735244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:30.735244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:30.735244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:30.735244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:30.737838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 178/358] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:30.739073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cdeef-c2b6-479a-a027-7ea80e51f472 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:30.787205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:30.793038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380ce1f-12e2-4319-8e5c-90e0183a1525 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:30.840841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff54ac-1b51-452a-b41f-164c919672da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:30.841489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff54ac-1b51-452a-b41f-164c919672da None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:30.842046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff54ac-1b51-452a-b41f-164c919672da None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:30.842317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff54ac-1b51-452a-b41f-164c919672da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:30.843152 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff54ac-1b51-452a-b41f-164c919672da None admin] RBAC: Adding query filter params (group.id=b3128d1154244e50a747a074c4abde20, role.id=efddf5f5222644b18b6836117621adf8, scope.project.id=1244700aec90459fbdbde9822e93a110) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:30.843468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff54ac-1b51-452a-b41f-164c919672da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:30.848997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff54ac-1b51-452a-b41f-164c919672da None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:30.853519 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 181/359] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 08:32:30 2022] GET /identity/v3/role_assignments?group.id=b3128d1154244e50a747a074c4abde20&scope.project.id=1244700aec90459fbdbde9822e93a110&role.id=efddf5f5222644b18b6836117621adf8 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:30.855157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff54ac-1b51-452a-b41f-164c919672da None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:30.877435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cdeef-c2b6-479a-a027-7ea80e51f472 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:30.918536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3e8ff-b0a3-42cc-b247-2a3530e9bce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:30.919350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3e8ff-b0a3-42cc-b247-2a3530e9bce3 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:30.919350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3e8ff-b0a3-42cc-b247-2a3530e9bce3 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:30.919636 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3e8ff-b0a3-42cc-b247-2a3530e9bce3 None admin] PATH_INFO: `/v3/groups/b3128d1154244e50a747a074c4abde20` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:30.921113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3e8ff-b0a3-42cc-b247-2a3530e9bce3 None admin] RBAC: Authorizing `identity:get_group(group_id=b3128d1154244e50a747a074c4abde20)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:30.922782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3e8ff-b0a3-42cc-b247-2a3530e9bce3 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:30.924973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 179/360] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:30 2022] GET /identity/v3/groups/b3128d1154244e50a747a074c4abde20 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:30.930507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3e8ff-b0a3-42cc-b247-2a3530e9bce3 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:30.990373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff54ac-1b51-452a-b41f-164c919672da None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:31.002216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:31.041569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a0908-b863-4b9e-9323-ed1c42ef59b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:31.042230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a0908-b863-4b9e-9323-ed1c42ef59b1 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:31.042514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a0908-b863-4b9e-9323-ed1c42ef59b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:31.042788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a0908-b863-4b9e-9323-ed1c42ef59b1 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:31.043466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a0908-b863-4b9e-9323-ed1c42ef59b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:31.043778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a0908-b863-4b9e-9323-ed1c42ef59b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:31.045163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a0908-b863-4b9e-9323-ed1c42ef59b1 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:31.060250 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 182/361] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:31.065736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d448a-7ad7-4ea4-9d93-28c7665f3c36 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:31.065736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d448a-7ad7-4ea4-9d93-28c7665f3c36 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:31.065736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d448a-7ad7-4ea4-9d93-28c7665f3c36 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:31.065736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 183/362] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:31.066043 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d448a-7ad7-4ea4-9d93-28c7665f3c36 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:31.073773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3e8ff-b0a3-42cc-b247-2a3530e9bce3 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:31.096358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:31.117450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7916c-7713-4d34-ad87-08902fbf7478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:31.118373 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7916c-7713-4d34-ad87-08902fbf7478 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:31.118651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7916c-7713-4d34-ad87-08902fbf7478 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:31.118980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7916c-7713-4d34-ad87-08902fbf7478 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:31.136874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7916c-7713-4d34-ad87-08902fbf7478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-81d7916c-7713-4d34-ad87-08902fbf7478 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:31.141819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:31.142687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:31.143469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 180/363] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:32:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:31.144142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7916c-7713-4d34-ad87-08902fbf7478 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:31.203174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d448a-7ad7-4ea4-9d93-28c7665f3c36 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:31.248137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c576f558-0997-477f-bdc1-7f864797814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:31.248740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576f558-0997-477f-bdc1-7f864797814e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:31.249014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576f558-0997-477f-bdc1-7f864797814e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:31.249279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576f558-0997-477f-bdc1-7f864797814e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:31.250700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576f558-0997-477f-bdc1-7f864797814e None admin] RBAC: Adding query filter params (group.id=b3128d1154244e50a747a074c4abde20, role.id=c0bf52b447564f769d707e96facb20a1, scope.project.id=1244700aec90459fbdbde9822e93a110) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:31.250700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576f558-0997-477f-bdc1-7f864797814e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:31.255404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c576f558-0997-477f-bdc1-7f864797814e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:31.260838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 184/364] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 08:32:31 2022] GET /identity/v3/role_assignments?group.id=b3128d1154244e50a747a074c4abde20&scope.project.id=1244700aec90459fbdbde9822e93a110&role.id=c0bf52b447564f769d707e96facb20a1 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:31.261912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e767b-ad8a-44f9-b186-d0bb02e85183 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:31.262254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e767b-ad8a-44f9-b186-d0bb02e85183 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:31.262497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e767b-ad8a-44f9-b186-d0bb02e85183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:31.276090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7916c-7713-4d34-ad87-08902fbf7478 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:31.283162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-430e767b-ad8a-44f9-b186-d0bb02e85183 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:31.284510 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-430e767b-ad8a-44f9-b186-d0bb02e85183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:31.289058 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:31.327278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030d838-3fdb-4666-b2ae-e756a3c01c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:31.327911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030d838-3fdb-4666-b2ae-e756a3c01c56 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:31.328188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030d838-3fdb-4666-b2ae-e756a3c01c56 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:31.328458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030d838-3fdb-4666-b2ae-e756a3c01c56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:31.372924 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 185/365] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:31.377997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e767b-ad8a-44f9-b186-d0bb02e85183 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:31.416027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:31.423723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030d838-3fdb-4666-b2ae-e756a3c01c56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:31.426268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d030d838-3fdb-4666-b2ae-e756a3c01c56 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:31.546026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e767b-ad8a-44f9-b186-d0bb02e85183 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:31.583099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3353c6-4125-4526-b563-aef648396c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:31.584160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3353c6-4125-4526-b563-aef648396c3e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:31.584612 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3353c6-4125-4526-b563-aef648396c3e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:31.585190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3353c6-4125-4526-b563-aef648396c3e None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:31.586061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3353c6-4125-4526-b563-aef648396c3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:31.586630 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3353c6-4125-4526-b563-aef648396c3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:31.588193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3353c6-4125-4526-b563-aef648396c3e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:31.597401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 186/366] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:31.603640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3353c6-4125-4526-b563-aef648396c3e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:31.620499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 181/367] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:31.624687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030d838-3fdb-4666-b2ae-e756a3c01c56 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:31.645372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:31.753189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:31.778067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3353c6-4125-4526-b563-aef648396c3e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:31.781363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d030d838-3fdb-4666-b2ae-e756a3c01c56 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:31.816114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49360b24-c490-4b1d-889e-b35678b60d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:31.816674 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49360b24-c490-4b1d-889e-b35678b60d56 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:31.816953 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49360b24-c490-4b1d-889e-b35678b60d56 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:31.817221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49360b24-c490-4b1d-889e-b35678b60d56 None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:31.818508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49360b24-c490-4b1d-889e-b35678b60d56 None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:31.820117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49360b24-c490-4b1d-889e-b35678b60d56 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:31.820240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2900c4-c160-4ebe-a02b-0096d3485606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:31.821117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2900c4-c160-4ebe-a02b-0096d3485606 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:31.821614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2900c4-c160-4ebe-a02b-0096d3485606 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:31.822106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2900c4-c160-4ebe-a02b-0096d3485606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:31.822205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 182/368] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:31 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:31.822960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49360b24-c490-4b1d-889e-b35678b60d56 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:31.899776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2900c4-c160-4ebe-a02b-0096d3485606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:31.901745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2900c4-c160-4ebe-a02b-0096d3485606 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:31.904046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:31.926336 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49360b24-c490-4b1d-889e-b35678b60d56 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:31.961033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b396b-e6f1-4ec2-a9fc-3b208c19367f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:31.961642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b396b-e6f1-4ec2-a9fc-3b208c19367f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:31.961948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b396b-e6f1-4ec2-a9fc-3b208c19367f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:31.962219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b396b-e6f1-4ec2-a9fc-3b208c19367f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:31.972632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b396b-e6f1-4ec2-a9fc-3b208c19367f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-cc2b396b-e6f1-4ec2-a9fc-3b208c19367f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:31.977552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:31.978436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:31.979470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 183/369] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:32:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:31.980132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b396b-e6f1-4ec2-a9fc-3b208c19367f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:31.999420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:32.023135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 187/370] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 08:32:32.024289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2900c4-c160-4ebe-a02b-0096d3485606 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:32.088051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b396b-e6f1-4ec2-a9fc-3b208c19367f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:32.123161 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a6162-e786-470e-959c-7018143d7ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:32.123815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a6162-e786-470e-959c-7018143d7ff1 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:32.124118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a6162-e786-470e-959c-7018143d7ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:32.124339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a6162-e786-470e-959c-7018143d7ff1 None admin] PATH_INFO: `/v3/groups/b3128d1154244e50a747a074c4abde20` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:32.125624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a6162-e786-470e-959c-7018143d7ff1 None admin] RBAC: Authorizing `identity:get_group(group_id=b3128d1154244e50a747a074c4abde20)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:32.126933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a6162-e786-470e-959c-7018143d7ff1 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:32.128489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 184/371] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:31 2022] GET /identity/v3/groups/b3128d1154244e50a747a074c4abde20 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:32.129444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a6162-e786-470e-959c-7018143d7ff1 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:32.136339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2900c4-c160-4ebe-a02b-0096d3485606 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:32.138491 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:32.176773 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190a2e7-32d5-4653-982e-6c7a25a47ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:32.177752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190a2e7-32d5-4653-982e-6c7a25a47ba4 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:32.178225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190a2e7-32d5-4653-982e-6c7a25a47ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:32.178728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190a2e7-32d5-4653-982e-6c7a25a47ba4 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:32.179749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190a2e7-32d5-4653-982e-6c7a25a47ba4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:32.180386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190a2e7-32d5-4653-982e-6c7a25a47ba4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:32.181988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190a2e7-32d5-4653-982e-6c7a25a47ba4 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:32.191278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 188/372] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:32.192458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190a2e7-32d5-4653-982e-6c7a25a47ba4 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:32.241974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a6162-e786-470e-959c-7018143d7ff1 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:32.244942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:32.277324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c75d21-8dcd-4c15-acc5-3dfd3d9f499d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:32.277987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c75d21-8dcd-4c15-acc5-3dfd3d9f499d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:32.278261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c75d21-8dcd-4c15-acc5-3dfd3d9f499d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:32.278587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c75d21-8dcd-4c15-acc5-3dfd3d9f499d None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:32.279732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c75d21-8dcd-4c15-acc5-3dfd3d9f499d None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:32.281492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c75d21-8dcd-4c15-acc5-3dfd3d9f499d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:32.283043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 185/373] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:32 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:32.283622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c75d21-8dcd-4c15-acc5-3dfd3d9f499d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:32.309986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190a2e7-32d5-4653-982e-6c7a25a47ba4 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:32.349033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f6283-b208-4925-bb33-124afc2313bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:32.350008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f6283-b208-4925-bb33-124afc2313bf None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:32.350499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f6283-b208-4925-bb33-124afc2313bf None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:32.351506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f6283-b208-4925-bb33-124afc2313bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:32.352423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f6283-b208-4925-bb33-124afc2313bf None admin] RBAC: Adding query filter params (group.id=b3128d1154244e50a747a074c4abde20, role.id=efddf5f5222644b18b6836117621adf8, scope.project.id=e2499b13100b44bb8d6160b53bc3747c) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:32.352989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f6283-b208-4925-bb33-124afc2313bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:32.354990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f6283-b208-4925-bb33-124afc2313bf None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:32.361683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 189/374] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 08:32:32 2022] GET /identity/v3/role_assignments?group.id=b3128d1154244e50a747a074c4abde20&scope.project.id=e2499b13100b44bb8d6160b53bc3747c&role.id=efddf5f5222644b18b6836117621adf8 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:32.362820 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f6283-b208-4925-bb33-124afc2313bf None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:32.364135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:32.392343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c75d21-8dcd-4c15-acc5-3dfd3d9f499d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:32.431967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba1402-ff6e-453e-b538-ec415bb981bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:32.432540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba1402-ff6e-453e-b538-ec415bb981bf None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:32.432816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba1402-ff6e-453e-b538-ec415bb981bf None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:32.433086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba1402-ff6e-453e-b538-ec415bb981bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:32.499657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f6283-b208-4925-bb33-124afc2313bf None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:32.504215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba1402-ff6e-453e-b538-ec415bb981bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:32.505750 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba1402-ff6e-453e-b538-ec415bb981bf None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:32.511229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:32.566983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00402ea7-84e8-47c1-8bbb-f459075b8ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:32.567686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00402ea7-84e8-47c1-8bbb-f459075b8ccf None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:32.567972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00402ea7-84e8-47c1-8bbb-f459075b8ccf None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:32.568244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00402ea7-84e8-47c1-8bbb-f459075b8ccf None admin] PATH_INFO: `/v3/groups/b3128d1154244e50a747a074c4abde20` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:32.573614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00402ea7-84e8-47c1-8bbb-f459075b8ccf None admin] RBAC: Authorizing `identity:get_group(group_id=b3128d1154244e50a747a074c4abde20)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:32.575129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00402ea7-84e8-47c1-8bbb-f459075b8ccf None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:32.579164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 190/375] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:32 2022] GET /identity/v3/groups/b3128d1154244e50a747a074c4abde20 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:32.586926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00402ea7-84e8-47c1-8bbb-f459075b8ccf None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:32.664727 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:32.700312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 186/376] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:32.707227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba1402-ff6e-453e-b538-ec415bb981bf None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:32.777569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00402ea7-84e8-47c1-8bbb-f459075b8ccf None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:32.838944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f5c1f-8189-4b17-bf2f-ffbd9827e7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:32.839559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f5c1f-8189-4b17-bf2f-ffbd9827e7ba None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:32.839838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f5c1f-8189-4b17-bf2f-ffbd9827e7ba None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:32.840110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f5c1f-8189-4b17-bf2f-ffbd9827e7ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:32.840848 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f5c1f-8189-4b17-bf2f-ffbd9827e7ba None admin] RBAC: Adding query filter params (group.id=b3128d1154244e50a747a074c4abde20, role.id=c0bf52b447564f769d707e96facb20a1, scope.project.id=e2499b13100b44bb8d6160b53bc3747c) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:32.841156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f5c1f-8189-4b17-bf2f-ffbd9827e7ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:32.842616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f5c1f-8189-4b17-bf2f-ffbd9827e7ba None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:32.858285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 191/377] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 08:32:32 2022] GET /identity/v3/role_assignments?group.id=b3128d1154244e50a747a074c4abde20&scope.project.id=e2499b13100b44bb8d6160b53bc3747c&role.id=c0bf52b447564f769d707e96facb20a1 => generated 260 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:32.884856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba1402-ff6e-453e-b538-ec415bb981bf None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:32.924076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a8267-9910-4ee9-8fd2-81d413833322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:32.924634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a8267-9910-4ee9-8fd2-81d413833322 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:32.924916 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a8267-9910-4ee9-8fd2-81d413833322 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:32.925191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a8267-9910-4ee9-8fd2-81d413833322 None admin] PATH_INFO: `/v3/projects/47b3fa8678ad4f779e78d8c72364fac2` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:32.927317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a8267-9910-4ee9-8fd2-81d413833322 None admin] RBAC: Authorizing `identity:get_project(project_id=47b3fa8678ad4f779e78d8c72364fac2)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:32.928896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a8267-9910-4ee9-8fd2-81d413833322 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:32.930932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 187/378] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:32 2022] GET /identity/v3/projects/47b3fa8678ad4f779e78d8c72364fac2 => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:32.936146 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f5c1f-8189-4b17-bf2f-ffbd9827e7ba None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:32.985751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:33.054618 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f5c1f-8189-4b17-bf2f-ffbd9827e7ba None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:33.098301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e2f07-265d-47e2-8f31-d75d255b641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:33.099038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e2f07-265d-47e2-8f31-d75d255b641f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:33.099325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e2f07-265d-47e2-8f31-d75d255b641f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:33.099598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e2f07-265d-47e2-8f31-d75d255b641f None admin] PATH_INFO: `/v3/groups/a35cb6ca12394f27ba3334bb9a386b9d` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:33.107204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e2f07-265d-47e2-8f31-d75d255b641f None admin] RBAC: Authorizing `identity:get_group(group_id=a35cb6ca12394f27ba3334bb9a386b9d)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:33.109077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e2f07-265d-47e2-8f31-d75d255b641f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:33.111349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 192/379] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:32 2022] GET /identity/v3/groups/a35cb6ca12394f27ba3334bb9a386b9d => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:33.117016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a8267-9910-4ee9-8fd2-81d413833322 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:33.182125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:33.249503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a8267-9910-4ee9-8fd2-81d413833322 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:33.292768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d26ea9-f7cb-4700-b3c2-817490591ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:33.293416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d26ea9-f7cb-4700-b3c2-817490591ef9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:33.293722 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d26ea9-f7cb-4700-b3c2-817490591ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:33.294002 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d26ea9-f7cb-4700-b3c2-817490591ef9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:33.294740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d26ea9-f7cb-4700-b3c2-817490591ef9 None admin] RBAC: Adding query filter params (group.id=a35cb6ca12394f27ba3334bb9a386b9d, role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=47b3fa8678ad4f779e78d8c72364fac2) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:33.295049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d26ea9-f7cb-4700-b3c2-817490591ef9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:33.296521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d26ea9-f7cb-4700-b3c2-817490591ef9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:33.305856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 188/380] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 08:32:33 2022] GET /identity/v3/role_assignments?group.id=a35cb6ca12394f27ba3334bb9a386b9d&scope.project.id=47b3fa8678ad4f779e78d8c72364fac2&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:33.461157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97aa31-c4c1-402b-868e-5f81bf16fa36 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:33.461157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97aa31-c4c1-402b-868e-5f81bf16fa36 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:33.461157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97aa31-c4c1-402b-868e-5f81bf16fa36 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:33.461863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 193/381] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:33.467630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e911e8-015e-4d6c-ba97-58ec4d671d5c None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:33.467913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e911e8-015e-4d6c-ba97-58ec4d671d5c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:33.468193 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e911e8-015e-4d6c-ba97-58ec4d671d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:33.496289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-40e911e8-015e-4d6c-ba97-58ec4d671d5c None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:33.497511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-40e911e8-015e-4d6c-ba97-58ec4d671d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:33.579241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 189/382] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:34.021503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97aa31-c4c1-402b-868e-5f81bf16fa36 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:34.031438 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e3b23-163b-40ea-b25f-fb90cf8e308c None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:34.031733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e3b23-163b-40ea-b25f-fb90cf8e308c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:34.032009 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e3b23-163b-40ea-b25f-fb90cf8e308c None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:34.034120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 190/383] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:34.040784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1093d31-0cb9-4199-a69a-50b3500229b3 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:34.041068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1093d31-0cb9-4199-a69a-50b3500229b3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:34.041335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1093d31-0cb9-4199-a69a-50b3500229b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:34.065253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f1093d31-0cb9-4199-a69a-50b3500229b3 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:34.066655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f1093d31-0cb9-4199-a69a-50b3500229b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:34.122146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:34.181780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 191/384] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:34.230394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97aa31-c4c1-402b-868e-5f81bf16fa36 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:34.233010 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:34.279655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352321d-4dee-4322-934a-892a79020ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:34.280210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352321d-4dee-4322-934a-892a79020ac4 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:34.280495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352321d-4dee-4322-934a-892a79020ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:34.280765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352321d-4dee-4322-934a-892a79020ac4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:34.303094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352321d-4dee-4322-934a-892a79020ac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-8352321d-4dee-4322-934a-892a79020ac4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:34.307983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:34.308828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:34.309603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 194/385] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:34.314945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1093d31-0cb9-4199-a69a-50b3500229b3 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:34.476830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1093d31-0cb9-4199-a69a-50b3500229b3 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:34.514595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352321d-4dee-4322-934a-892a79020ac4 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:34.519785 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ace17b-2afc-49c1-ab86-5f50c5be66fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:34.520777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ace17b-2afc-49c1-ab86-5f50c5be66fb None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:34.521292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ace17b-2afc-49c1-ab86-5f50c5be66fb None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:34.521874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ace17b-2afc-49c1-ab86-5f50c5be66fb None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:34.522798 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ace17b-2afc-49c1-ab86-5f50c5be66fb None admin] RBAC: Adding query filter params (name=member) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:34.523292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ace17b-2afc-49c1-ab86-5f50c5be66fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:34.524848 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ace17b-2afc-49c1-ab86-5f50c5be66fb None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:34.534273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 192/386] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:34.546754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ace17b-2afc-49c1-ab86-5f50c5be66fb None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:34.615907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:34.702319 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352321d-4dee-4322-934a-892a79020ac4 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:34.742320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ace17b-2afc-49c1-ab86-5f50c5be66fb None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:34.766927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f296b-e091-40cb-aff1-006c74e9d107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:34.768786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f296b-e091-40cb-aff1-006c74e9d107 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:34.769063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f296b-e091-40cb-aff1-006c74e9d107 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:34.769335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f296b-e091-40cb-aff1-006c74e9d107 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:34.788788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f296b-e091-40cb-aff1-006c74e9d107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-b70f296b-e091-40cb-aff1-006c74e9d107 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:34.795065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:34.795933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:34.796723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 195/387] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:32:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:34.804523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f296b-e091-40cb-aff1-006c74e9d107 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:34.810429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cff29-aac3-4992-bf79-46aaa0f8cfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:34.811071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cff29-aac3-4992-bf79-46aaa0f8cfc5 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:34.811351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cff29-aac3-4992-bf79-46aaa0f8cfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:34.811621 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cff29-aac3-4992-bf79-46aaa0f8cfc5 None admin] PATH_INFO: `/v3/groups/b3128d1154244e50a747a074c4abde20` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:34.813704 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32cff29-aac3-4992-bf79-46aaa0f8cfc5 None admin] RBAC: Authorizing `identity:get_group(group_id=b3128d1154244e50a747a074c4abde20)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:34.815332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32cff29-aac3-4992-bf79-46aaa0f8cfc5 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:34.821375 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 193/388] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:34 2022] GET /identity/v3/groups/b3128d1154244e50a747a074c4abde20 => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:34.822215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cff29-aac3-4992-bf79-46aaa0f8cfc5 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:34.856356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:35.002389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:35.003992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f296b-e091-40cb-aff1-006c74e9d107 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:35.013646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cff29-aac3-4992-bf79-46aaa0f8cfc5 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:35.068429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac927274-f6b2-4101-8ba8-61f50835984e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:35.068969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac927274-f6b2-4101-8ba8-61f50835984e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:35.069791 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac927274-f6b2-4101-8ba8-61f50835984e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:35.070070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac927274-f6b2-4101-8ba8-61f50835984e None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:35.070961 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac927274-f6b2-4101-8ba8-61f50835984e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:35.071359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac927274-f6b2-4101-8ba8-61f50835984e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:35.072786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac927274-f6b2-4101-8ba8-61f50835984e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:35.076120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59577542-9d0e-4465-ac5c-e7b6f23c6c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:35.077209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59577542-9d0e-4465-ac5c-e7b6f23c6c44 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:35.077494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59577542-9d0e-4465-ac5c-e7b6f23c6c44 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:35.077793 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59577542-9d0e-4465-ac5c-e7b6f23c6c44 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:35.081381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59577542-9d0e-4465-ac5c-e7b6f23c6c44 None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:35.082876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59577542-9d0e-4465-ac5c-e7b6f23c6c44 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:35.086106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 194/389] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:34 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:35.094800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59577542-9d0e-4465-ac5c-e7b6f23c6c44 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:35.096218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 196/390] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:35.102022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac927274-f6b2-4101-8ba8-61f50835984e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:35.238056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:35.260966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac927274-f6b2-4101-8ba8-61f50835984e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:35.271926 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59577542-9d0e-4465-ac5c-e7b6f23c6c44 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:35.307892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a486b40-67ed-41be-baab-3d8d9c6bf67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:35.308525 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a486b40-67ed-41be-baab-3d8d9c6bf67c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:35.308804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a486b40-67ed-41be-baab-3d8d9c6bf67c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:35.309138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a486b40-67ed-41be-baab-3d8d9c6bf67c None admin] PATH_INFO: `/v3/groups/b3128d1154244e50a747a074c4abde20` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:35.310926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a486b40-67ed-41be-baab-3d8d9c6bf67c None admin] RBAC: Authorizing `identity:get_group(group_id=b3128d1154244e50a747a074c4abde20)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:35.312830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a486b40-67ed-41be-baab-3d8d9c6bf67c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:35.317743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 197/391] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:35 2022] GET /identity/v3/groups/b3128d1154244e50a747a074c4abde20 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:35.321522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2479827-ff37-4115-b2e3-1795144219ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:35.322091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7973b9a-f924-45d5-8b0c-a573c1a78393 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:35.322451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7973b9a-f924-45d5-8b0c-a573c1a78393 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:35.322759 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7973b9a-f924-45d5-8b0c-a573c1a78393 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:35.323786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2479827-ff37-4115-b2e3-1795144219ba None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:35.324286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2479827-ff37-4115-b2e3-1795144219ba None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:35.328463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 198/392] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:35.328463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7973b9a-f924-45d5-8b0c-a573c1a78393 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:35.329107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2479827-ff37-4115-b2e3-1795144219ba None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110/groups/b3128d1154244e50a747a074c4abde20/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:35.333325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2479827-ff37-4115-b2e3-1795144219ba None admin] RBAC: Authorizing `identity:create_grant(project_id=1244700aec90459fbdbde9822e93a110, group_id=b3128d1154244e50a747a074c4abde20, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:35.335457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2479827-ff37-4115-b2e3-1795144219ba None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:35.354949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 195/393] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Oct 4 08:32:35 2022] PUT /identity/v3/projects/1244700aec90459fbdbde9822e93a110/groups/b3128d1154244e50a747a074c4abde20/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:32:35.357555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ccd94-54c2-44bb-9166-13ea47a2de1b None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:35.358116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ccd94-54c2-44bb-9166-13ea47a2de1b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:35.358636 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ccd94-54c2-44bb-9166-13ea47a2de1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:35.383247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-149ccd94-54c2-44bb-9166-13ea47a2de1b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:35.385673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-149ccd94-54c2-44bb-9166-13ea47a2de1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:35.409226 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:35.465570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7973b9a-f924-45d5-8b0c-a573c1a78393 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:35.473836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 196/394] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:35.511506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dac3b-8ee2-48de-a5ba-50118b247fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:35.512149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dac3b-8ee2-48de-a5ba-50118b247fbb None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:35.512424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dac3b-8ee2-48de-a5ba-50118b247fbb None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:35.512698 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dac3b-8ee2-48de-a5ba-50118b247fbb None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:35.513913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67dac3b-8ee2-48de-a5ba-50118b247fbb None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:35.515403 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67dac3b-8ee2-48de-a5ba-50118b247fbb None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:35.517162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 199/395] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:35 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:35.523587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ccd94-54c2-44bb-9166-13ea47a2de1b None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:35.639927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ccd94-54c2-44bb-9166-13ea47a2de1b None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:35.685133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb684e1d-a1d2-4725-8303-2a48c98ed1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:35.686267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb684e1d-a1d2-4725-8303-2a48c98ed1fe None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:35.686749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb684e1d-a1d2-4725-8303-2a48c98ed1fe None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:35.687231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb684e1d-a1d2-4725-8303-2a48c98ed1fe None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110/groups/b3128d1154244e50a747a074c4abde20/roles/c0bf52b447564f769d707e96facb20a1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:35.696226 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb684e1d-a1d2-4725-8303-2a48c98ed1fe None admin] RBAC: Authorizing `identity:create_grant(project_id=1244700aec90459fbdbde9822e93a110, group_id=b3128d1154244e50a747a074c4abde20, role_id=c0bf52b447564f769d707e96facb20a1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:35.698246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb684e1d-a1d2-4725-8303-2a48c98ed1fe None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:35.714983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 197/396] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Oct 4 08:32:35 2022] PUT /identity/v3/projects/1244700aec90459fbdbde9822e93a110/groups/b3128d1154244e50a747a074c4abde20/roles/c0bf52b447564f769d707e96facb20a1 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:32:35.787871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dac3b-8ee2-48de-a5ba-50118b247fbb None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:35.817082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:35.930197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dac3b-8ee2-48de-a5ba-50118b247fbb None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:35.975167 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfe543-4625-4ea9-825f-edf1b3fd3a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:35.976713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfe543-4625-4ea9-825f-edf1b3fd3a7d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:35.977202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfe543-4625-4ea9-825f-edf1b3fd3a7d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:35.977680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfe543-4625-4ea9-825f-edf1b3fd3a7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:35.989878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfe543-4625-4ea9-825f-edf1b3fd3a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-04dfe543-4625-4ea9-825f-edf1b3fd3a7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:35.997353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:35.998333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:35.999163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:35.999163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:35.999163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:35.999163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:35.999163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:35.999163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:35.999163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:35.999163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:35.999163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:35.999163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:35.999163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:35.999163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:36.001794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 200/397] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:36.007708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb684e1d-a1d2-4725-8303-2a48c98ed1fe None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:36.024430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:36.035410 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab4e7f-94ee-49b2-8159-570d2a0924b7 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:36.035712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab4e7f-94ee-49b2-8159-570d2a0924b7 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:36.035959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab4e7f-94ee-49b2-8159-570d2a0924b7 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:36.037573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 201/398] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:36.042958 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66562ec2-eb97-44e2-8c6a-6658a2328a2d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:36.043243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66562ec2-eb97-44e2-8c6a-6658a2328a2d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:36.043517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66562ec2-eb97-44e2-8c6a-6658a2328a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:36.067037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-66562ec2-eb97-44e2-8c6a-6658a2328a2d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:36.067840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-66562ec2-eb97-44e2-8c6a-6658a2328a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:36.133330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb684e1d-a1d2-4725-8303-2a48c98ed1fe None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:36.168157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 202/399] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:36.173936 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763e70c-27ae-4566-a1f6-fbf40dfd3696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:36.174730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763e70c-27ae-4566-a1f6-fbf40dfd3696 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:36.175153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763e70c-27ae-4566-a1f6-fbf40dfd3696 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:36.175580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763e70c-27ae-4566-a1f6-fbf40dfd3696 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:36.176378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763e70c-27ae-4566-a1f6-fbf40dfd3696 None admin] RBAC: Adding query filter params (name=member) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:36.176836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763e70c-27ae-4566-a1f6-fbf40dfd3696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:36.178372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763e70c-27ae-4566-a1f6-fbf40dfd3696 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:36.187971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 198/400] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:36.193758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66562ec2-eb97-44e2-8c6a-6658a2328a2d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:36.203176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f9252-3018-4feb-8080-e73fd935799e None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:36.203463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f9252-3018-4feb-8080-e73fd935799e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:36.203739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f9252-3018-4feb-8080-e73fd935799e None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:36.205305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 199/401] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:32:36.210704 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84902d0e-f855-4076-ac58-9d8d046abc01 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:36.210986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84902d0e-f855-4076-ac58-9d8d046abc01 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:36.211425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84902d0e-f855-4076-ac58-9d8d046abc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:36.215612 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:36.232012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-84902d0e-f855-4076-ac58-9d8d046abc01 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:36.233200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-84902d0e-f855-4076-ac58-9d8d046abc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:36.314972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 200/402] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:36.388403 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66562ec2-eb97-44e2-8c6a-6658a2328a2d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:36.422678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d57f3-f491-4999-9a32-caa99122327a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:36.423315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d57f3-f491-4999-9a32-caa99122327a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:36.423596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d57f3-f491-4999-9a32-caa99122327a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:36.423871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d57f3-f491-4999-9a32-caa99122327a None admin] PATH_INFO: `/v3/groups/b3128d1154244e50a747a074c4abde20` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:36.426608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d57f3-f491-4999-9a32-caa99122327a None admin] RBAC: Authorizing `identity:get_group(group_id=b3128d1154244e50a747a074c4abde20)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:36.428068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d57f3-f491-4999-9a32-caa99122327a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:36.430535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 203/403] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:36 2022] GET /identity/v3/groups/b3128d1154244e50a747a074c4abde20 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:36.435675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84902d0e-f855-4076-ac58-9d8d046abc01 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:36.448159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:36.512634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d57f3-f491-4999-9a32-caa99122327a None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:36.587298 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84902d0e-f855-4076-ac58-9d8d046abc01 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:36.608391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:36.628219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e18b0-dc4a-4e30-b6f7-b86c53f502e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:36.628858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e18b0-dc4a-4e30-b6f7-b86c53f502e2 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:36.629133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e18b0-dc4a-4e30-b6f7-b86c53f502e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:36.629403 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e18b0-dc4a-4e30-b6f7-b86c53f502e2 None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:36.629669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d57f3-f491-4999-9a32-caa99122327a None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:36.630558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e18b0-dc4a-4e30-b6f7-b86c53f502e2 None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:36.632705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e18b0-dc4a-4e30-b6f7-b86c53f502e2 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:36.635611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 201/404] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:36 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:36.644482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e18b0-dc4a-4e30-b6f7-b86c53f502e2 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:36.719519 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0d178-aeb2-48be-bb3f-d5ec8ec69a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:36.720075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0d178-aeb2-48be-bb3f-d5ec8ec69a68 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:36.720355 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0d178-aeb2-48be-bb3f-d5ec8ec69a68 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:36.720628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0d178-aeb2-48be-bb3f-d5ec8ec69a68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:36.739786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0d178-aeb2-48be-bb3f-d5ec8ec69a68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-7ed0d178-aeb2-48be-bb3f-d5ec8ec69a68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:36.745142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:36.746085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:36.746859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:36.746859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:36.746859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:36.746859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:36.746859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:36.746859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:36.746859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:36.746859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:36.746859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:36.746859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:36.746859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:36.746859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:36.747960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 204/405] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:32:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:32:36.754705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0d178-aeb2-48be-bb3f-d5ec8ec69a68 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:36.782069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:36.815951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e18b0-dc4a-4e30-b6f7-b86c53f502e2 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:36.858105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fbc20-6cbf-4a62-b48e-58681f520eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:36.858767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fbc20-6cbf-4a62-b48e-58681f520eaa None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:36.859043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fbc20-6cbf-4a62-b48e-58681f520eaa None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:36.859311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819fbc20-6cbf-4a62-b48e-58681f520eaa None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c/groups/b3128d1154244e50a747a074c4abde20/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:36.861199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819fbc20-6cbf-4a62-b48e-58681f520eaa None admin] RBAC: Authorizing `identity:create_grant(project_id=e2499b13100b44bb8d6160b53bc3747c, group_id=b3128d1154244e50a747a074c4abde20, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:36.862662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819fbc20-6cbf-4a62-b48e-58681f520eaa None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:36.880644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 202/406] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Oct 4 08:32:36 2022] PUT /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c/groups/b3128d1154244e50a747a074c4abde20/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:32:36.881678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fbc20-6cbf-4a62-b48e-58681f520eaa None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:36.894597 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0d178-aeb2-48be-bb3f-d5ec8ec69a68 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:36.942464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25472594-ea60-4598-8a96-56727f3372da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:36.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472594-ea60-4598-8a96-56727f3372da None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:36.944106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472594-ea60-4598-8a96-56727f3372da None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:36.944951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472594-ea60-4598-8a96-56727f3372da None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:36.946057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25472594-ea60-4598-8a96-56727f3372da None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:36.946478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25472594-ea60-4598-8a96-56727f3372da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:36.949606 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25472594-ea60-4598-8a96-56727f3372da None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:36.963122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 205/407] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:36.968991 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25472594-ea60-4598-8a96-56727f3372da None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:36.972974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:37.015047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819fbc20-6cbf-4a62-b48e-58681f520eaa None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:37.056556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ccd50-1c5f-4c15-b245-0e8d37061ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:37.057418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ccd50-1c5f-4c15-b245-0e8d37061ff1 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:37.057879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ccd50-1c5f-4c15-b245-0e8d37061ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:37.058256 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ccd50-1c5f-4c15-b245-0e8d37061ff1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:37.060592 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:37.076323 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ccd50-1c5f-4c15-b245-0e8d37061ff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-ab0ccd50-1c5f-4c15-b245-0e8d37061ff1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:37.090709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:37.091661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:37.092499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 203/408] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:32:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:37.113760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ccd50-1c5f-4c15-b245-0e8d37061ff1 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:37.158761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25472594-ea60-4598-8a96-56727f3372da None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:37.217370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7806ff-d989-4ea6-aa17-82e6bedb7bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:37.218505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7806ff-d989-4ea6-aa17-82e6bedb7bae None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:37.218977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7806ff-d989-4ea6-aa17-82e6bedb7bae None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:37.219439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7806ff-d989-4ea6-aa17-82e6bedb7bae None admin] PATH_INFO: `/v3/groups/a35cb6ca12394f27ba3334bb9a386b9d` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:37.220908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7806ff-d989-4ea6-aa17-82e6bedb7bae None admin] RBAC: Authorizing `identity:get_group(group_id=a35cb6ca12394f27ba3334bb9a386b9d)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:37.222693 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7806ff-d989-4ea6-aa17-82e6bedb7bae None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:37.230912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ccd50-1c5f-4c15-b245-0e8d37061ff1 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:37.231426 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 206/409] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:36 2022] GET /identity/v3/groups/a35cb6ca12394f27ba3334bb9a386b9d => generated 229 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:37.237211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7806ff-d989-4ea6-aa17-82e6bedb7bae None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:37.271740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a510aa-2b3c-4f2e-b305-c49299501fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:37.273320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a510aa-2b3c-4f2e-b305-c49299501fb3 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:37.273788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a510aa-2b3c-4f2e-b305-c49299501fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:37.274096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a510aa-2b3c-4f2e-b305-c49299501fb3 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:37.274764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a510aa-2b3c-4f2e-b305-c49299501fb3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:37.275325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a510aa-2b3c-4f2e-b305-c49299501fb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:37.277404 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a510aa-2b3c-4f2e-b305-c49299501fb3 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:37.287050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 204/410] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:37.292757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a510aa-2b3c-4f2e-b305-c49299501fb3 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:37.325510 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:37.359728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:37.372469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7806ff-d989-4ea6-aa17-82e6bedb7bae None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:37.431225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a510aa-2b3c-4f2e-b305-c49299501fb3 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:37.441921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09d3d4-6081-46ff-8cb8-a41d15c12d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:37.442569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09d3d4-6081-46ff-8cb8-a41d15c12d55 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:37.442846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09d3d4-6081-46ff-8cb8-a41d15c12d55 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:37.443122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09d3d4-6081-46ff-8cb8-a41d15c12d55 None admin] PATH_INFO: `/v3/projects/47b3fa8678ad4f779e78d8c72364fac2` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:37.444799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09d3d4-6081-46ff-8cb8-a41d15c12d55 None admin] RBAC: Authorizing `identity:get_project(project_id=47b3fa8678ad4f779e78d8c72364fac2)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:37.446291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09d3d4-6081-46ff-8cb8-a41d15c12d55 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:37.448035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 207/411] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:37 2022] GET /identity/v3/projects/47b3fa8678ad4f779e78d8c72364fac2 => generated 344 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:37.455121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09d3d4-6081-46ff-8cb8-a41d15c12d55 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:37.501420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb105fa1-5c60-4981-951e-7fa290679091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:37.502003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb105fa1-5c60-4981-951e-7fa290679091 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:37.502282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb105fa1-5c60-4981-951e-7fa290679091 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:37.502553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb105fa1-5c60-4981-951e-7fa290679091 None admin] PATH_INFO: `/v3/groups/b3128d1154244e50a747a074c4abde20` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:37.505743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb105fa1-5c60-4981-951e-7fa290679091 None admin] RBAC: Authorizing `identity:get_group(group_id=b3128d1154244e50a747a074c4abde20)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:37.507764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb105fa1-5c60-4981-951e-7fa290679091 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:37.509352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 205/412] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:37 2022] GET /identity/v3/groups/b3128d1154244e50a747a074c4abde20 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:37.515633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb105fa1-5c60-4981-951e-7fa290679091 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:37.619536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09d3d4-6081-46ff-8cb8-a41d15c12d55 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:37.656559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa29514-879d-4096-bcba-64c5e152ae5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:37.657342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa29514-879d-4096-bcba-64c5e152ae5c None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:37.657849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa29514-879d-4096-bcba-64c5e152ae5c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:37.658152 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa29514-879d-4096-bcba-64c5e152ae5c None admin] PATH_INFO: `/v3/projects/47b3fa8678ad4f779e78d8c72364fac2/groups/a35cb6ca12394f27ba3334bb9a386b9d/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:37.660239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa29514-879d-4096-bcba-64c5e152ae5c None admin] RBAC: Authorizing `identity:create_grant(project_id=47b3fa8678ad4f779e78d8c72364fac2, group_id=a35cb6ca12394f27ba3334bb9a386b9d, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:37.661896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa29514-879d-4096-bcba-64c5e152ae5c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:37.673000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:37.684069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 208/413] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Oct 4 08:32:37 2022] PUT /identity/v3/projects/47b3fa8678ad4f779e78d8c72364fac2/groups/a35cb6ca12394f27ba3334bb9a386b9d/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:32:37.724804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb105fa1-5c60-4981-951e-7fa290679091 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:37.754698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:37.764428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670cb78-e45a-4d60-8387-707e6491a7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:37.765090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670cb78-e45a-4d60-8387-707e6491a7e9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:37.765368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670cb78-e45a-4d60-8387-707e6491a7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:37.765646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670cb78-e45a-4d60-8387-707e6491a7e9 None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:37.766983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670cb78-e45a-4d60-8387-707e6491a7e9 None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:37.768566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670cb78-e45a-4d60-8387-707e6491a7e9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:37.770641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 206/414] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:37 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:37.779103 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa29514-879d-4096-bcba-64c5e152ae5c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:37.927829 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa29514-879d-4096-bcba-64c5e152ae5c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:37.969044 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decfd5e-b2f1-487f-8954-20c996b72093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:37.969723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decfd5e-b2f1-487f-8954-20c996b72093 None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:37.970015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decfd5e-b2f1-487f-8954-20c996b72093 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:37.970290 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decfd5e-b2f1-487f-8954-20c996b72093 None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c/groups/b3128d1154244e50a747a074c4abde20/roles/c0bf52b447564f769d707e96facb20a1` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:37.972488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decfd5e-b2f1-487f-8954-20c996b72093 None admin] RBAC: Authorizing `identity:create_grant(project_id=e2499b13100b44bb8d6160b53bc3747c, group_id=b3128d1154244e50a747a074c4abde20, role_id=c0bf52b447564f769d707e96facb20a1)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:37.973970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decfd5e-b2f1-487f-8954-20c996b72093 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:37.989727 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 209/415] 10.222.0.26 () {58 vars in 1442 bytes} [Tue Oct 4 08:32:37 2022] PUT /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c/groups/b3128d1154244e50a747a074c4abde20/roles/c0bf52b447564f769d707e96facb20a1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:32:38.058793 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f370bf2-e58e-4e95-b2d5-2f7a22257104 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:38.059127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f370bf2-e58e-4e95-b2d5-2f7a22257104 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:38.059398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f370bf2-e58e-4e95-b2d5-2f7a22257104 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:38.060906 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 207/416] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:38.067092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fd851-abcf-48b1-99b5-81e2ac02095b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:38.067774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fd851-abcf-48b1-99b5-81e2ac02095b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:38.068248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fd851-abcf-48b1-99b5-81e2ac02095b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:38.089204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-412fd851-abcf-48b1-99b5-81e2ac02095b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:38.090775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-412fd851-abcf-48b1-99b5-81e2ac02095b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:38.164852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:38.187280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 210/417] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:38.579150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f370bf2-e58e-4e95-b2d5-2f7a22257104 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:38.645915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0fffc-79ad-4966-9f54-9de7094b7df3 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:38.646421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0fffc-79ad-4966-9f54-9de7094b7df3 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:38.646493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0fffc-79ad-4966-9f54-9de7094b7df3 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:38.650519 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 211/418] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:38.655057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533520b-0476-492f-8d78-d8a80b055d8f None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:38.655452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533520b-0476-492f-8d78-d8a80b055d8f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:38.655827 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533520b-0476-492f-8d78-d8a80b055d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:38.675790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-a533520b-0476-492f-8d78-d8a80b055d8f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:38.677124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a533520b-0476-492f-8d78-d8a80b055d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:38.769775 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f370bf2-e58e-4e95-b2d5-2f7a22257104 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:38.781846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 212/419] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:38.804697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc31b1-6d71-4ac8-b5c8-0dfb03461e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:38.805551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc31b1-6d71-4ac8-b5c8-0dfb03461e06 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:38.805996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc31b1-6d71-4ac8-b5c8-0dfb03461e06 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:38.806424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc31b1-6d71-4ac8-b5c8-0dfb03461e06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:38.816663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc31b1-6d71-4ac8-b5c8-0dfb03461e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-23dc31b1-6d71-4ac8-b5c8-0dfb03461e06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:38.822520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:38.823590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:38.824562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:38.824562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:38.824562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:38.824562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:38.824562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:38.824562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:38.824562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:38.824562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:38.824562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:38.824562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:38.824562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:38.824562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:38.826726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 208/420] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:38.832607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533520b-0476-492f-8d78-d8a80b055d8f None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:38.965996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533520b-0476-492f-8d78-d8a80b055d8f None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:38.994298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:39.019243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396cdae8-e97d-4e94-84db-f5ce6501ce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:39.020111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cdae8-e97d-4e94-84db-f5ce6501ce16 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:39.020543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cdae8-e97d-4e94-84db-f5ce6501ce16 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:39.020952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396cdae8-e97d-4e94-84db-f5ce6501ce16 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:39.021775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396cdae8-e97d-4e94-84db-f5ce6501ce16 None admin] RBAC: Adding query filter params (name=member) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:39.022241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396cdae8-e97d-4e94-84db-f5ce6501ce16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:39.023823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396cdae8-e97d-4e94-84db-f5ce6501ce16 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:39.036927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 213/421] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:39.043241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc31b1-6d71-4ac8-b5c8-0dfb03461e06 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:39.089876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396cdae8-e97d-4e94-84db-f5ce6501ce16 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:39.106866 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:39.189792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc31b1-6d71-4ac8-b5c8-0dfb03461e06 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:39.248248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f30fb5-6c0f-45fe-92fb-ad11055f2295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:39.248911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f30fb5-6c0f-45fe-92fb-ad11055f2295 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:39.249195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f30fb5-6c0f-45fe-92fb-ad11055f2295 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:39.249468 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f30fb5-6c0f-45fe-92fb-ad11055f2295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:39.280877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396cdae8-e97d-4e94-84db-f5ce6501ce16 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:39.346645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcbf72-2d64-47a0-948a-1b3601ec7596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:39.347197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcbf72-2d64-47a0-948a-1b3601ec7596 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:39.347472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcbf72-2d64-47a0-948a-1b3601ec7596 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:39.348334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcbf72-2d64-47a0-948a-1b3601ec7596 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:39.358076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f30fb5-6c0f-45fe-92fb-ad11055f2295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:39.359979 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f30fb5-6c0f-45fe-92fb-ad11055f2295 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:39.372170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcbf72-2d64-47a0-948a-1b3601ec7596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-b7fcbf72-2d64-47a0-948a-1b3601ec7596 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:39.377486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:39.378399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:39.379205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 214/422] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:32:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:32:39.387157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcbf72-2d64-47a0-948a-1b3601ec7596 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:39.457456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:39.466030 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:39.561494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 209/423] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:39.569830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f30fb5-6c0f-45fe-92fb-ad11055f2295 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:39.579299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcbf72-2d64-47a0-948a-1b3601ec7596 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:39.646449 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78865cee-5e02-40b1-8901-9ece6c3ebee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:39.647047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78865cee-5e02-40b1-8901-9ece6c3ebee3 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:39.647307 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78865cee-5e02-40b1-8901-9ece6c3ebee3 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:39.647578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78865cee-5e02-40b1-8901-9ece6c3ebee3 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:39.648254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78865cee-5e02-40b1-8901-9ece6c3ebee3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:39.648560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78865cee-5e02-40b1-8901-9ece6c3ebee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:39.649981 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78865cee-5e02-40b1-8901-9ece6c3ebee3 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:39.665000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 215/424] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:39.672935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78865cee-5e02-40b1-8901-9ece6c3ebee3 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:39.750364 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f30fb5-6c0f-45fe-92fb-ad11055f2295 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:39.790660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ca9f1-a0f8-44d8-9291-8fb8e8444e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:39.791650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ca9f1-a0f8-44d8-9291-8fb8e8444e92 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:39.792124 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ca9f1-a0f8-44d8-9291-8fb8e8444e92 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:39.792708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ca9f1-a0f8-44d8-9291-8fb8e8444e92 None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:39.794069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ca9f1-a0f8-44d8-9291-8fb8e8444e92 None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:39.795800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ca9f1-a0f8-44d8-9291-8fb8e8444e92 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:39.798033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 210/425] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:39 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:39.799665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca1eec-e523-4b73-ae89-e6c29ef32530 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:39.800160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca1eec-e523-4b73-ae89-e6c29ef32530 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:39.800627 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca1eec-e523-4b73-ae89-e6c29ef32530 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:39.802147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 211/426] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:39.803197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca1eec-e523-4b73-ae89-e6c29ef32530 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:39.813273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78865cee-5e02-40b1-8901-9ece6c3ebee3 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:39.815499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:39.827889 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:39.860837 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1b9b0-81aa-4571-9236-f3790da3d584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:39.861803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1b9b0-81aa-4571-9236-f3790da3d584 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:39.862287 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1b9b0-81aa-4571-9236-f3790da3d584 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:39.863255 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1b9b0-81aa-4571-9236-f3790da3d584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:39.915378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca1eec-e523-4b73-ae89-e6c29ef32530 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:39.947719 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1b9b0-81aa-4571-9236-f3790da3d584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:39.950153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1b9b0-81aa-4571-9236-f3790da3d584 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:39.954214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71dca2-5cf8-4a2b-878c-a65c2c5a9529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:39.955172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71dca2-5cf8-4a2b-878c-a65c2c5a9529 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:39.955647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71dca2-5cf8-4a2b-878c-a65c2c5a9529 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:39.956114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71dca2-5cf8-4a2b-878c-a65c2c5a9529 None admin] PATH_INFO: `/v3/groups/b3128d1154244e50a747a074c4abde20` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:39.957422 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d71dca2-5cf8-4a2b-878c-a65c2c5a9529 None admin] RBAC: Authorizing `identity:get_group(group_id=b3128d1154244e50a747a074c4abde20)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:39.959261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d71dca2-5cf8-4a2b-878c-a65c2c5a9529 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:39.961276 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 212/427] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:39 2022] GET /identity/v3/groups/b3128d1154244e50a747a074c4abde20 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:39.962779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27be2c-e74c-4f84-baba-fa806c630970 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:39.963260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27be2c-e74c-4f84-baba-fa806c630970 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:39.963765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27be2c-e74c-4f84-baba-fa806c630970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:39.983492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-5d27be2c-e74c-4f84-baba-fa806c630970 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:39.985101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5d27be2c-e74c-4f84-baba-fa806c630970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:40.065816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 216/428] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:40.066728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1b9b0-81aa-4571-9236-f3790da3d584 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:40.069084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:40.072270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 213/429] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:40.073368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27be2c-e74c-4f84-baba-fa806c630970 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:40.095367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:40.196806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27be2c-e74c-4f84-baba-fa806c630970 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:40.241410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa2bb2-6394-4d41-ae2a-38c4392e7a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:40.242629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa2bb2-6394-4d41-ae2a-38c4392e7a3d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:40.243639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa2bb2-6394-4d41-ae2a-38c4392e7a3d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:40.244139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa2bb2-6394-4d41-ae2a-38c4392e7a3d None admin] PATH_INFO: `/v3/projects/1244700aec90459fbdbde9822e93a110` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:40.245927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa2bb2-6394-4d41-ae2a-38c4392e7a3d None admin] RBAC: Authorizing `identity:get_project(project_id=1244700aec90459fbdbde9822e93a110)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:40.248422 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa2bb2-6394-4d41-ae2a-38c4392e7a3d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:40.255594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1b9b0-81aa-4571-9236-f3790da3d584 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:40.256011 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 214/430] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:40 2022] GET /identity/v3/projects/1244700aec90459fbdbde9822e93a110 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:40.258406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa2bb2-6394-4d41-ae2a-38c4392e7a3d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:40.293192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9ec9f-ee65-4d5a-999d-6eadbf07a759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:40.293947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9ec9f-ee65-4d5a-999d-6eadbf07a759 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:40.294372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9ec9f-ee65-4d5a-999d-6eadbf07a759 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:40.294732 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9ec9f-ee65-4d5a-999d-6eadbf07a759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:40.295718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9ec9f-ee65-4d5a-999d-6eadbf07a759 None admin] RBAC: Adding query filter params (group.id=b3128d1154244e50a747a074c4abde20, role.id=efddf5f5222644b18b6836117621adf8, scope.project.id=1244700aec90459fbdbde9822e93a110) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:40.296329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9ec9f-ee65-4d5a-999d-6eadbf07a759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:40.297854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9ec9f-ee65-4d5a-999d-6eadbf07a759 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:40.304621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 217/431] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 08:32:40 2022] GET /identity/v3/role_assignments?group.id=b3128d1154244e50a747a074c4abde20&scope.project.id=1244700aec90459fbdbde9822e93a110&role.id=efddf5f5222644b18b6836117621adf8 => generated 611 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:40.310171 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49880728-fc88-4c2e-a143-4bd83af46ab3 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:40.310596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49880728-fc88-4c2e-a143-4bd83af46ab3 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:40.310993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49880728-fc88-4c2e-a143-4bd83af46ab3 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:40.312797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 218/432] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:40.318313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961597b3-44b6-425c-8802-009af4d07559 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:40.318697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961597b3-44b6-425c-8802-009af4d07559 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:40.319069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961597b3-44b6-425c-8802-009af4d07559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:40.358542 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-961597b3-44b6-425c-8802-009af4d07559 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:40.360088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-961597b3-44b6-425c-8802-009af4d07559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:40.366750 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:40.385526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa2bb2-6394-4d41-ae2a-38c4392e7a3d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:40.429575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779647a3-87e7-47dd-9179-e3f5607ded34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:40.430561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779647a3-87e7-47dd-9179-e3f5607ded34 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:40.430983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779647a3-87e7-47dd-9179-e3f5607ded34 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:40.431395 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779647a3-87e7-47dd-9179-e3f5607ded34 None admin] PATH_INFO: `/v3/groups/b3128d1154244e50a747a074c4abde20` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:40.433294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779647a3-87e7-47dd-9179-e3f5607ded34 None admin] RBAC: Authorizing `identity:get_group(group_id=b3128d1154244e50a747a074c4abde20)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:40.435162 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779647a3-87e7-47dd-9179-e3f5607ded34 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:40.442097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 215/433] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:40 2022] GET /identity/v3/groups/b3128d1154244e50a747a074c4abde20 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:40.445665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779647a3-87e7-47dd-9179-e3f5607ded34 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:40.474639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:40.484856 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 219/434] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:40.485750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961597b3-44b6-425c-8802-009af4d07559 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:40.589507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779647a3-87e7-47dd-9179-e3f5607ded34 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:40.611392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961597b3-44b6-425c-8802-009af4d07559 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:40.637747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a79c86-6f10-40af-9f25-1c3172bb9b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:40.638634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a79c86-6f10-40af-9f25-1c3172bb9b61 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:40.639006 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a79c86-6f10-40af-9f25-1c3172bb9b61 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:40.639375 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a79c86-6f10-40af-9f25-1c3172bb9b61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:40.640202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a79c86-6f10-40af-9f25-1c3172bb9b61 None admin] RBAC: Adding query filter params (group.id=b3128d1154244e50a747a074c4abde20, role.id=c0bf52b447564f769d707e96facb20a1, scope.project.id=1244700aec90459fbdbde9822e93a110) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:40.640610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a79c86-6f10-40af-9f25-1c3172bb9b61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:40.642372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a79c86-6f10-40af-9f25-1c3172bb9b61 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:40.651611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 216/435] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 08:32:40 2022] GET /identity/v3/role_assignments?group.id=b3128d1154244e50a747a074c4abde20&scope.project.id=1244700aec90459fbdbde9822e93a110&role.id=c0bf52b447564f769d707e96facb20a1 => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:40.667745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90235a28-4cf8-4f33-8b23-f3a8478cf2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:40.667745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90235a28-4cf8-4f33-8b23-f3a8478cf2c6 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:40.667745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90235a28-4cf8-4f33-8b23-f3a8478cf2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:40.667745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90235a28-4cf8-4f33-8b23-f3a8478cf2c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:40.684817 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90235a28-4cf8-4f33-8b23-f3a8478cf2c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-90235a28-4cf8-4f33-8b23-f3a8478cf2c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:40.689971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:40.690906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:40.691705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 220/436] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:40.698013 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a79c86-6f10-40af-9f25-1c3172bb9b61 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:40.721575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:40.739873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aea10-ea05-4974-a5af-e34da3bed7be None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:40.740129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aea10-ea05-4974-a5af-e34da3bed7be None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:40.740447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aea10-ea05-4974-a5af-e34da3bed7be None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:40.743893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 221/437] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:40.750495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea82d8-a629-4df5-9adc-62aeb281b700 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:40.750495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea82d8-a629-4df5-9adc-62aeb281b700 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:40.750495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea82d8-a629-4df5-9adc-62aeb281b700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:40.777727 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-44ea82d8-a629-4df5-9adc-62aeb281b700 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:40.779281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-44ea82d8-a629-4df5-9adc-62aeb281b700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:40.838519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a79c86-6f10-40af-9f25-1c3172bb9b61 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:40.853190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:40.879735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 222/438] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:40.881142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea82d8-a629-4df5-9adc-62aeb281b700 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:40.886339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c2e3f-34f0-4bf1-8a88-bee2ea9551b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:40.887265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c2e3f-34f0-4bf1-8a88-bee2ea9551b4 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:40.887744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c2e3f-34f0-4bf1-8a88-bee2ea9551b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:40.888238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c2e3f-34f0-4bf1-8a88-bee2ea9551b4 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:40.889107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c2e3f-34f0-4bf1-8a88-bee2ea9551b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:40.889611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c2e3f-34f0-4bf1-8a88-bee2ea9551b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:40.891258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c2e3f-34f0-4bf1-8a88-bee2ea9551b4 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:40.901792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 217/439] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:32:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:40.906853 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c2e3f-34f0-4bf1-8a88-bee2ea9551b4 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:41.010310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:41.015206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea82d8-a629-4df5-9adc-62aeb281b700 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:41.033390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c2e3f-34f0-4bf1-8a88-bee2ea9551b4 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:41.061453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0195d20-6f22-4bed-9b8f-a6cef57b37d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:41.062125 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0195d20-6f22-4bed-9b8f-a6cef57b37d4 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:41.062408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0195d20-6f22-4bed-9b8f-a6cef57b37d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:41.062682 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0195d20-6f22-4bed-9b8f-a6cef57b37d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:41.073653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0195d20-6f22-4bed-9b8f-a6cef57b37d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:41.075852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8060579-ba26-4cc6-9628-6e42693e429e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:41.076453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8060579-ba26-4cc6-9628-6e42693e429e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:41.076729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8060579-ba26-4cc6-9628-6e42693e429e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:41.076996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8060579-ba26-4cc6-9628-6e42693e429e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-c0195d20-6f22-4bed-9b8f-a6cef57b37d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:41.078648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:41.079495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:41.080408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 223/440] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:32:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:41.085407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0195d20-6f22-4bed-9b8f-a6cef57b37d4 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:41.143528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:41.166087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8060579-ba26-4cc6-9628-6e42693e429e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:41.167980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8060579-ba26-4cc6-9628-6e42693e429e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:41.205680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0195d20-6f22-4bed-9b8f-a6cef57b37d4 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:41.244040 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0d5f6-cd8b-4b4b-8492-081dc36e9785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:41.244945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0d5f6-cd8b-4b4b-8492-081dc36e9785 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:41.244945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0d5f6-cd8b-4b4b-8492-081dc36e9785 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:41.245227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0d5f6-cd8b-4b4b-8492-081dc36e9785 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:41.245929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0d5f6-cd8b-4b4b-8492-081dc36e9785 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:41.246240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0d5f6-cd8b-4b4b-8492-081dc36e9785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:41.248179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0d5f6-cd8b-4b4b-8492-081dc36e9785 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:41.260306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 224/441] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:41.260306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0d5f6-cd8b-4b4b-8492-081dc36e9785 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:41.263703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:41.288334 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 218/442] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 08:32:41.289408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8060579-ba26-4cc6-9628-6e42693e429e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:41.369981 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0d5f6-cd8b-4b4b-8492-081dc36e9785 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:41.378492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:41.390671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8060579-ba26-4cc6-9628-6e42693e429e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:41.403201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de7f14-ca5f-4492-aba2-9dccde1fa217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:41.403814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de7f14-ca5f-4492-aba2-9dccde1fa217 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:41.404089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de7f14-ca5f-4492-aba2-9dccde1fa217 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:41.404357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de7f14-ca5f-4492-aba2-9dccde1fa217 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:41.414098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de7f14-ca5f-4492-aba2-9dccde1fa217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-f6de7f14-ca5f-4492-aba2-9dccde1fa217 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:41.419153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:41.420069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:41.420842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 225/443] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:32:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:41.422608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de7f14-ca5f-4492-aba2-9dccde1fa217 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:41.425224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a337f-c2a5-46ac-a5d3-649f059ea5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:41.425863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a337f-c2a5-46ac-a5d3-649f059ea5db None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:41.426147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a337f-c2a5-46ac-a5d3-649f059ea5db None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:41.426444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a337f-c2a5-46ac-a5d3-649f059ea5db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:41.473428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:41.494943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a337f-c2a5-46ac-a5d3-649f059ea5db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:41.496406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a337f-c2a5-46ac-a5d3-649f059ea5db None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:41.520995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de7f14-ca5f-4492-aba2-9dccde1fa217 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:41.554170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e34d66-3d8b-45c7-bedd-5e9db57e6e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:41.554782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e34d66-3d8b-45c7-bedd-5e9db57e6e4d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:41.555054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e34d66-3d8b-45c7-bedd-5e9db57e6e4d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:41.555324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e34d66-3d8b-45c7-bedd-5e9db57e6e4d None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:41.556432 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e34d66-3d8b-45c7-bedd-5e9db57e6e4d None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:41.557956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e34d66-3d8b-45c7-bedd-5e9db57e6e4d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:41.559545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 226/444] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:41 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:41.560291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e34d66-3d8b-45c7-bedd-5e9db57e6e4d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:41.594401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:41.598888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 219/445] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:41.599558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a337f-c2a5-46ac-a5d3-649f059ea5db None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:41.659824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e34d66-3d8b-45c7-bedd-5e9db57e6e4d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:41.682269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:41.694708 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c90c4d-0a84-4f24-8038-3b71084d79a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:41.695317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c90c4d-0a84-4f24-8038-3b71084d79a9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:41.695597 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c90c4d-0a84-4f24-8038-3b71084d79a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:41.695873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c90c4d-0a84-4f24-8038-3b71084d79a9 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:41.696536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c90c4d-0a84-4f24-8038-3b71084d79a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:41.696841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c90c4d-0a84-4f24-8038-3b71084d79a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:41.698270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c90c4d-0a84-4f24-8038-3b71084d79a9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:41.701444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a337f-c2a5-46ac-a5d3-649f059ea5db None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:41.706080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 227/446] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:41.706865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c90c4d-0a84-4f24-8038-3b71084d79a9 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:41.738531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568b41-a062-4ad2-b35f-3c64165f24d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:41.739178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568b41-a062-4ad2-b35f-3c64165f24d8 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:41.739453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568b41-a062-4ad2-b35f-3c64165f24d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:41.739723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568b41-a062-4ad2-b35f-3c64165f24d8 None admin] PATH_INFO: `/v3/groups/b3128d1154244e50a747a074c4abde20` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:41.740811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77568b41-a062-4ad2-b35f-3c64165f24d8 None admin] RBAC: Authorizing `identity:get_group(group_id=b3128d1154244e50a747a074c4abde20)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:41.742271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77568b41-a062-4ad2-b35f-3c64165f24d8 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:41.743806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 220/447] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:41 2022] GET /identity/v3/groups/b3128d1154244e50a747a074c4abde20 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:41.744547 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568b41-a062-4ad2-b35f-3c64165f24d8 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:41.805779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:41.809403 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c90c4d-0a84-4f24-8038-3b71084d79a9 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:41.839955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568b41-a062-4ad2-b35f-3c64165f24d8 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:41.843138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de5c0d-78b8-461d-8b59-0894cca6b488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:41.843757 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de5c0d-78b8-461d-8b59-0894cca6b488 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:41.844038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de5c0d-78b8-461d-8b59-0894cca6b488 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:41.844311 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de5c0d-78b8-461d-8b59-0894cca6b488 None admin] PATH_INFO: `/v3/projects/47b3fa8678ad4f779e78d8c72364fac2` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:41.845426 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de5c0d-78b8-461d-8b59-0894cca6b488 None admin] RBAC: Authorizing `identity:get_project(project_id=47b3fa8678ad4f779e78d8c72364fac2)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:41.846914 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de5c0d-78b8-461d-8b59-0894cca6b488 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:41.848508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 228/448] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:41 2022] GET /identity/v3/projects/47b3fa8678ad4f779e78d8c72364fac2 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:41.849270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de5c0d-78b8-461d-8b59-0894cca6b488 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:41.877810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f56bee-1378-4d1b-ab19-5df1897449a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:41.878407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f56bee-1378-4d1b-ab19-5df1897449a7 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:41.878682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f56bee-1378-4d1b-ab19-5df1897449a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:41.878952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f56bee-1378-4d1b-ab19-5df1897449a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:41.898713 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:41.947303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de5c0d-78b8-461d-8b59-0894cca6b488 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:41.948212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f56bee-1378-4d1b-ab19-5df1897449a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:41.949672 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f56bee-1378-4d1b-ab19-5df1897449a7 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:41.981602 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97793da3-8e62-477b-a454-0583035e54bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:41.982242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97793da3-8e62-477b-a454-0583035e54bc None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:41.982522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97793da3-8e62-477b-a454-0583035e54bc None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:41.982792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97793da3-8e62-477b-a454-0583035e54bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:41.983514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97793da3-8e62-477b-a454-0583035e54bc None admin] RBAC: Adding query filter params (group.id=b3128d1154244e50a747a074c4abde20, role.id=efddf5f5222644b18b6836117621adf8, scope.project.id=e2499b13100b44bb8d6160b53bc3747c) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:41.983822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97793da3-8e62-477b-a454-0583035e54bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:41.985261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97793da3-8e62-477b-a454-0583035e54bc None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:41.990846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 229/449] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 08:32:41 2022] GET /identity/v3/role_assignments?group.id=b3128d1154244e50a747a074c4abde20&scope.project.id=e2499b13100b44bb8d6160b53bc3747c&role.id=efddf5f5222644b18b6836117621adf8 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:41.992201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97793da3-8e62-477b-a454-0583035e54bc None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:42.024812 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:42.071373 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 221/450] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:42.076736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f56bee-1378-4d1b-ab19-5df1897449a7 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:42.104233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97793da3-8e62-477b-a454-0583035e54bc None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:42.130373 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:42.141634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e33c6b-981e-4619-b2fe-e042394b4656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:42.142313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e33c6b-981e-4619-b2fe-e042394b4656 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:42.142594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e33c6b-981e-4619-b2fe-e042394b4656 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:42.142874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e33c6b-981e-4619-b2fe-e042394b4656 None admin] PATH_INFO: `/v3/groups/a35cb6ca12394f27ba3334bb9a386b9d` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:42.144034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e33c6b-981e-4619-b2fe-e042394b4656 None admin] RBAC: Authorizing `identity:get_group(group_id=a35cb6ca12394f27ba3334bb9a386b9d)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:42.145498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e33c6b-981e-4619-b2fe-e042394b4656 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:42.147152 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 230/451] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:41 2022] GET /identity/v3/groups/a35cb6ca12394f27ba3334bb9a386b9d => generated 229 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:42.153645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e33c6b-981e-4619-b2fe-e042394b4656 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:42.194713 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f56bee-1378-4d1b-ab19-5df1897449a7 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:42.240356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9c9df-e349-415b-ab41-cba71eabb95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:42.241611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9c9df-e349-415b-ab41-cba71eabb95e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:42.242025 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9c9df-e349-415b-ab41-cba71eabb95e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:42.242432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9c9df-e349-415b-ab41-cba71eabb95e None admin] PATH_INFO: `/v3/projects/e2499b13100b44bb8d6160b53bc3747c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:42.243756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9c9df-e349-415b-ab41-cba71eabb95e None admin] RBAC: Authorizing `identity:get_project(project_id=e2499b13100b44bb8d6160b53bc3747c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:42.245267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9c9df-e349-415b-ab41-cba71eabb95e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:42.247143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 222/452] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:32:42 2022] GET /identity/v3/projects/e2499b13100b44bb8d6160b53bc3747c => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:42.254090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9c9df-e349-415b-ab41-cba71eabb95e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:42.268356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:42.274743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e33c6b-981e-4619-b2fe-e042394b4656 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:42.317441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14148d-e8e1-474f-aa38-a0b2b5da14c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:42.318089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14148d-e8e1-474f-aa38-a0b2b5da14c1 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:42.318370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14148d-e8e1-474f-aa38-a0b2b5da14c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:42.318643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14148d-e8e1-474f-aa38-a0b2b5da14c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:42.319879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd14148d-e8e1-474f-aa38-a0b2b5da14c1 None admin] RBAC: Adding query filter params (group.id=a35cb6ca12394f27ba3334bb9a386b9d, role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=47b3fa8678ad4f779e78d8c72364fac2) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:42.320282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd14148d-e8e1-474f-aa38-a0b2b5da14c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:42.322352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd14148d-e8e1-474f-aa38-a0b2b5da14c1 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:42.328811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 231/453] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 08:32:42 2022] GET /identity/v3/role_assignments?group.id=a35cb6ca12394f27ba3334bb9a386b9d&scope.project.id=47b3fa8678ad4f779e78d8c72364fac2&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:42.373251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9c9df-e349-415b-ab41-cba71eabb95e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:42.391576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:42.411084 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10407e9b-b175-43a3-901c-baf20199d2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:42.411732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10407e9b-b175-43a3-901c-baf20199d2f8 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:42.412012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10407e9b-b175-43a3-901c-baf20199d2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:42.412283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10407e9b-b175-43a3-901c-baf20199d2f8 None admin] PATH_INFO: `/v3/groups/b3128d1154244e50a747a074c4abde20` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:42.413502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10407e9b-b175-43a3-901c-baf20199d2f8 None admin] RBAC: Authorizing `identity:get_group(group_id=b3128d1154244e50a747a074c4abde20)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:42.415014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10407e9b-b175-43a3-901c-baf20199d2f8 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:42.416620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 223/454] 10.222.0.26 () {56 vars in 1180 bytes} [Tue Oct 4 08:32:42 2022] GET /identity/v3/groups/b3128d1154244e50a747a074c4abde20 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:42.422646 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14148d-e8e1-474f-aa38-a0b2b5da14c1 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:42.528636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14148d-e8e1-474f-aa38-a0b2b5da14c1 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:42.562223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968138c4-12b4-4818-98de-1e6253941416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:42.562818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968138c4-12b4-4818-98de-1e6253941416 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:42.563099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968138c4-12b4-4818-98de-1e6253941416 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:42.563373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968138c4-12b4-4818-98de-1e6253941416 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:42.564095 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968138c4-12b4-4818-98de-1e6253941416 None admin] RBAC: Adding query filter params (group.id=b3128d1154244e50a747a074c4abde20, role.id=c0bf52b447564f769d707e96facb20a1, scope.project.id=e2499b13100b44bb8d6160b53bc3747c) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:42.564404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968138c4-12b4-4818-98de-1e6253941416 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:42.565844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968138c4-12b4-4818-98de-1e6253941416 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:42.572008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 232/455] 10.222.0.26 () {56 vars in 1376 bytes} [Tue Oct 4 08:32:42 2022] GET /identity/v3/role_assignments?group.id=b3128d1154244e50a747a074c4abde20&scope.project.id=e2499b13100b44bb8d6160b53bc3747c&role.id=c0bf52b447564f769d707e96facb20a1 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:45.266787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e336c43-9b3e-445a-ba32-6c448d6d5679 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:45.267869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e336c43-9b3e-445a-ba32-6c448d6d5679 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:45.268409 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e336c43-9b3e-445a-ba32-6c448d6d5679 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:45.270244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 224/456] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:45.275599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a2d13-e9b4-422b-83b2-c39b2a43c8ea None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:45.276411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a2d13-e9b4-422b-83b2-c39b2a43c8ea None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:45.276894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a2d13-e9b4-422b-83b2-c39b2a43c8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:45.298650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-a73a2d13-e9b4-422b-83b2-c39b2a43c8ea None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:45.300153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a73a2d13-e9b4-422b-83b2-c39b2a43c8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:45.324157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:45.384705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 233/457] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:45.586428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ff08e-7ce1-4028-802a-e0558867df50 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:45.587073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ff08e-7ce1-4028-802a-e0558867df50 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:45.587550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ff08e-7ce1-4028-802a-e0558867df50 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:45.589302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 225/458] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:45.595085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b60f3-1598-42d1-8171-9d3759dc03d9 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:45.595607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b60f3-1598-42d1-8171-9d3759dc03d9 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:45.596075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b60f3-1598-42d1-8171-9d3759dc03d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:45.621831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-8c1b60f3-1598-42d1-8171-9d3759dc03d9 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:45.623334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8c1b60f3-1598-42d1-8171-9d3759dc03d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:45.676645 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b6b04-82a7-420f-aad9-1e3705c33eb8 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:45.677047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b6b04-82a7-420f-aad9-1e3705c33eb8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:45.677431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b6b04-82a7-420f-aad9-1e3705c33eb8 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:45.679810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 226/459] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:45.692141 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b6b04-82a7-420f-aad9-1e3705c33eb8 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:45.723684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 234/460] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:45.724945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b38fbf-4e01-4e86-b560-7ea81d491bad None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:45.725784 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b38fbf-4e01-4e86-b560-7ea81d491bad None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:45.726067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b38fbf-4e01-4e86-b560-7ea81d491bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:45.750928 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-42b38fbf-4e01-4e86-b560-7ea81d491bad None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:45.752697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-42b38fbf-4e01-4e86-b560-7ea81d491bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:45.832586 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 235/461] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:45.852069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b6b04-82a7-420f-aad9-1e3705c33eb8 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:45.903456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef189075-7915-4b57-aba1-cf7a071533a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:45.904320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef189075-7915-4b57-aba1-cf7a071533a9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:45.904743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef189075-7915-4b57-aba1-cf7a071533a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:45.905187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef189075-7915-4b57-aba1-cf7a071533a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:45.925156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef189075-7915-4b57-aba1-cf7a071533a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:45.926823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef189075-7915-4b57-aba1-cf7a071533a9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-ef189075-7915-4b57-aba1-cf7a071533a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:45.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:45.944484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:45.945577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:45.945577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:45.945577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:45.947273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 227/462] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:45.952916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b38fbf-4e01-4e86-b560-7ea81d491bad None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:45.971930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:46.063840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b38fbf-4e01-4e86-b560-7ea81d491bad None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:46.066442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef189075-7915-4b57-aba1-cf7a071533a9 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:46.105352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a908ff-5601-43e6-be05-28156f052dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:46.106554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a908ff-5601-43e6-be05-28156f052dea None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:46.107165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a908ff-5601-43e6-be05-28156f052dea None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:46.107769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a908ff-5601-43e6-be05-28156f052dea None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:46.108993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a908ff-5601-43e6-be05-28156f052dea None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:46.109662 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a908ff-5601-43e6-be05-28156f052dea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:46.112026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a908ff-5601-43e6-be05-28156f052dea None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:46.130125 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 236/463] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:46.136149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a908ff-5601-43e6-be05-28156f052dea None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:46.153390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:46.240703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef189075-7915-4b57-aba1-cf7a071533a9 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:46.248128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:46.273943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a908ff-5601-43e6-be05-28156f052dea None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:46.279734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e191e1-4bae-4792-ab08-c80ec826adbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:46.280907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e191e1-4bae-4792-ab08-c80ec826adbd None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:46.281337 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e191e1-4bae-4792-ab08-c80ec826adbd None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:46.281770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e191e1-4bae-4792-ab08-c80ec826adbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:46.293312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e191e1-4bae-4792-ab08-c80ec826adbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:46.294976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e191e1-4bae-4792-ab08-c80ec826adbd None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:46.310324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531d8d1-90e7-4357-97fd-ef3d7f70f7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:46.310937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531d8d1-90e7-4357-97fd-ef3d7f70f7e9 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:46.311227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531d8d1-90e7-4357-97fd-ef3d7f70f7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:46.311507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531d8d1-90e7-4357-97fd-ef3d7f70f7e9 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:46.312165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531d8d1-90e7-4357-97fd-ef3d7f70f7e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-b9e191e1-4bae-4792-ab08-c80ec826adbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:46.312407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:46.313451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:46.314362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:46.314362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:46.314362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:46.314517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531d8d1-90e7-4357-97fd-ef3d7f70f7e9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:46.315065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 228/464] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:32:46.315657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e191e1-4bae-4792-ab08-c80ec826adbd None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:46.346383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 237/465] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:32:46 2022] POST /identity/v3/users => generated 246 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:32:46.347924 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531d8d1-90e7-4357-97fd-ef3d7f70f7e9 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:46.436714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:46.471998 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e191e1-4bae-4792-ab08-c80ec826adbd None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:46.520109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf2b89-d1dc-4f90-8d62-bd25cce137ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:46.521191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf2b89-d1dc-4f90-8d62-bd25cce137ee None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:46.521683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf2b89-d1dc-4f90-8d62-bd25cce137ee None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:46.522246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf2b89-d1dc-4f90-8d62-bd25cce137ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:46.536072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf2b89-d1dc-4f90-8d62-bd25cce137ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:46.538602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf2b89-d1dc-4f90-8d62-bd25cce137ee None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:46.554326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531d8d1-90e7-4357-97fd-ef3d7f70f7e9 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-adaf2b89-d1dc-4f90-8d62-bd25cce137ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:46.557918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:46.558782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:46.559607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:46.559607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:46.559607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:46.561393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 229/466] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:46.563007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356d79-f368-4bf8-9cf3-ffca9ae828a4 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:46.563517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356d79-f368-4bf8-9cf3-ffca9ae828a4 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:46.564002 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356d79-f368-4bf8-9cf3-ffca9ae828a4 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:46.565905 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 230/467] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:46.567409 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20356d79-f368-4bf8-9cf3-ffca9ae828a4 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:46.591667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:46.601507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494070a7-8533-47bb-8606-3890c9be3931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:46.602478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494070a7-8533-47bb-8606-3890c9be3931 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:46.602943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494070a7-8533-47bb-8606-3890c9be3931 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:46.603352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494070a7-8533-47bb-8606-3890c9be3931 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:46.604460 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494070a7-8533-47bb-8606-3890c9be3931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:46.604762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494070a7-8533-47bb-8606-3890c9be3931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:46.606179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494070a7-8533-47bb-8606-3890c9be3931 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:46.622220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 238/468] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:46.623692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d0056-0aa0-43a7-9089-949358b3ce54 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:46.623979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d0056-0aa0-43a7-9089-949358b3ce54 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:46.624251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d0056-0aa0-43a7-9089-949358b3ce54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:46.651922 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b93d0056-0aa0-43a7-9089-949358b3ce54 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:46.653410 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b93d0056-0aa0-43a7-9089-949358b3ce54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:46.719576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20356d79-f368-4bf8-9cf3-ffca9ae828a4 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:46.741360 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 239/469] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:46.742487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d0056-0aa0-43a7-9089-949358b3ce54 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:46.769838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a7d50-9733-41a6-b0d9-23ba2d46fb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:46.770712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a7d50-9733-41a6-b0d9-23ba2d46fb36 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:46.771129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a7d50-9733-41a6-b0d9-23ba2d46fb36 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:46.771562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a7d50-9733-41a6-b0d9-23ba2d46fb36 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:46.773435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a7d50-9733-41a6-b0d9-23ba2d46fb36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:46.773936 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a7d50-9733-41a6-b0d9-23ba2d46fb36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:46.775938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a7d50-9733-41a6-b0d9-23ba2d46fb36 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:46.786059 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:46.791212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 231/470] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:46.797766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a7d50-9733-41a6-b0d9-23ba2d46fb36 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:46.917089 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a7d50-9733-41a6-b0d9-23ba2d46fb36 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:46.922512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93d0056-0aa0-43a7-9089-949358b3ce54 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:46.952461 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:46.962130 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeed384-4dac-4b56-ba76-65d7b4b5f605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:46.962964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeed384-4dac-4b56-ba76-65d7b4b5f605 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:46.963475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeed384-4dac-4b56-ba76-65d7b4b5f605 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:46.963950 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeed384-4dac-4b56-ba76-65d7b4b5f605 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:46.964921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeed384-4dac-4b56-ba76-65d7b4b5f605 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:46.966619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774200e9-2545-4115-82ca-5b0a8d5ee61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:46.966996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeed384-4dac-4b56-ba76-65d7b4b5f605 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:46.967278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774200e9-2545-4115-82ca-5b0a8d5ee61e None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:46.967564 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774200e9-2545-4115-82ca-5b0a8d5ee61e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:46.967897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774200e9-2545-4115-82ca-5b0a8d5ee61e None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:46.968539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774200e9-2545-4115-82ca-5b0a8d5ee61e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:46.970108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774200e9-2545-4115-82ca-5b0a8d5ee61e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:46.994960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 232/471] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:32:46 2022] POST /identity/v3/users => generated 248 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:32:46.996504 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e5ccd-3340-4ef6-98d2-7f6a6b3c8ece None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:46.996927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e5ccd-3340-4ef6-98d2-7f6a6b3c8ece None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:46.997332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e5ccd-3340-4ef6-98d2-7f6a6b3c8ece None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:47.003036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 233/472] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:47.004214 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 240/473] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:32:46 2022] POST /identity/v3/users => generated 248 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:32:47.007340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bc95c-4f90-47b7-84c1-428f9f628a2b None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:47.007616 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bc95c-4f90-47b7-84c1-428f9f628a2b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:47.007888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bc95c-4f90-47b7-84c1-428f9f628a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:47.050132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-9d2bc95c-4f90-47b7-84c1-428f9f628a2b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:47.051354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9d2bc95c-4f90-47b7-84c1-428f9f628a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:47.096508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774200e9-2545-4115-82ca-5b0a8d5ee61e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:47.124408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:47.145117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 234/474] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:47.276044 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774200e9-2545-4115-82ca-5b0a8d5ee61e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:47.313903 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bfa58-c967-4176-9c88-2892700114fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:47.314507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bfa58-c967-4176-9c88-2892700114fd None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:47.314785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bfa58-c967-4176-9c88-2892700114fd None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:47.315053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34bfa58-c967-4176-9c88-2892700114fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:47.327216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bfa58-c967-4176-9c88-2892700114fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:47.328757 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34bfa58-c967-4176-9c88-2892700114fd None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-f34bfa58-c967-4176-9c88-2892700114fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:47.343163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:47.344080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:47.345332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:47.345332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:47.345332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:47.345332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 241/475] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:47.351125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bc95c-4f90-47b7-84c1-428f9f628a2b None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:47.465085 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bc95c-4f90-47b7-84c1-428f9f628a2b None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:47.476460 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bfa58-c967-4176-9c88-2892700114fd None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:47.515970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aed20-2eb3-4173-8ed4-b800cae31a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:47.516581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aed20-2eb3-4173-8ed4-b800cae31a09 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:47.516853 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aed20-2eb3-4173-8ed4-b800cae31a09 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:47.517120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950aed20-2eb3-4173-8ed4-b800cae31a09 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:47.520641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950aed20-2eb3-4173-8ed4-b800cae31a09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:47.520845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950aed20-2eb3-4173-8ed4-b800cae31a09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:47.522725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950aed20-2eb3-4173-8ed4-b800cae31a09 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:47.542164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:47.544045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 235/476] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:47.552954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aed20-2eb3-4173-8ed4-b800cae31a09 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:47.636119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:47.702711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34bfa58-c967-4176-9c88-2892700114fd None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:47.709403 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950aed20-2eb3-4173-8ed4-b800cae31a09 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:47.754095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7b068-ae19-42d8-b09b-082431ab076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:47.755067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7b068-ae19-42d8-b09b-082431ab076b None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:47.755536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7b068-ae19-42d8-b09b-082431ab076b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:47.756015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7b068-ae19-42d8-b09b-082431ab076b None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:47.757209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7b068-ae19-42d8-b09b-082431ab076b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:47.759289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7b068-ae19-42d8-b09b-082431ab076b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:47.787571 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 236/477] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:32:47 2022] POST /identity/v3/users => generated 249 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:32:47.790089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f2cb5-30cd-4d9e-9d03-957c03309fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:47.790933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f2cb5-30cd-4d9e-9d03-957c03309fa9 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:47.791407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f2cb5-30cd-4d9e-9d03-957c03309fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:47.791800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f2cb5-30cd-4d9e-9d03-957c03309fa9 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:47.792691 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f2cb5-30cd-4d9e-9d03-957c03309fa9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:47.794262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f2cb5-30cd-4d9e-9d03-957c03309fa9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:47.812929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.sql.core [None req-c88f2cb5-30cd-4d9e-9d03-957c03309fa9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 08:32:47.812929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 08:32:47.812929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [parameters: {'id': '6ea0259ccb8a43b4a068cf6769562be4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 08:32:47.812929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87945) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-c88f2cb5-30cd-4d9e-9d03-957c03309fa9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application conn.query(q) Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application result.read() Oct 04 08:32:47.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return ref.to_dict() Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 08:32:47.838657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application next(self.gen) Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application yield resource Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application next(self.gen) Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise self.value Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application session.commit() Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 08:32:47.840233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self.session.flush() Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application compat.raise_( Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 08:32:47.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 08:32:47.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application conn.query(q) Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application result.read() Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 08:32:47.844704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [parameters: {'id': '6ea0259ccb8a43b4a068cf6769562be4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:47.846795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 08:32:47.848739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:47.851376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 242/478] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Oct 4 08:32:47 2022] POST /identity/v3/roles => generated 144 bytes in 363 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:47.851511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7b068-ae19-42d8-b09b-082431ab076b None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:47.967815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7b068-ae19-42d8-b09b-082431ab076b None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:47.969411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:48.005926 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0386116-84ee-4212-a205-79443ad7441b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:48.007175 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0386116-84ee-4212-a205-79443ad7441b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:48.007462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0386116-84ee-4212-a205-79443ad7441b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:48.007737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0386116-84ee-4212-a205-79443ad7441b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:48.019541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0386116-84ee-4212-a205-79443ad7441b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-d0386116-84ee-4212-a205-79443ad7441b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:48.025816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:48.027918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:48.029940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 237/479] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:32:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:48.036239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f2cb5-30cd-4d9e-9d03-957c03309fa9 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:48.145982 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:48.154018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f2cb5-30cd-4d9e-9d03-957c03309fa9 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:48.191683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047147f-f849-45a6-8231-d816ed70d554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:48.192297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047147f-f849-45a6-8231-d816ed70d554 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:48.192637 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047147f-f849-45a6-8231-d816ed70d554 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:48.192917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047147f-f849-45a6-8231-d816ed70d554 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:48.193591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047147f-f849-45a6-8231-d816ed70d554 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:48.193987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047147f-f849-45a6-8231-d816ed70d554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:48.195876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047147f-f849-45a6-8231-d816ed70d554 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:48.204410 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 243/480] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:48.860715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e02fad-48e4-4902-83b1-d45136292dc4 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:48.861258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e02fad-48e4-4902-83b1-d45136292dc4 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:48.861258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e02fad-48e4-4902-83b1-d45136292dc4 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:48.862848 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 238/481] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:48.875218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9f02c-3a64-4a50-9ec0-5f695ac8edc5 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:48.875635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9f02c-3a64-4a50-9ec0-5f695ac8edc5 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:48.875804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9f02c-3a64-4a50-9ec0-5f695ac8edc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:48.897912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-17d9f02c-3a64-4a50-9ec0-5f695ac8edc5 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:48.899277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-17d9f02c-3a64-4a50-9ec0-5f695ac8edc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:48.992015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 244/482] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:49.421382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e02fad-48e4-4902-83b1-d45136292dc4 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:49.561356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e02fad-48e4-4902-83b1-d45136292dc4 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:49.605018 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ea014-d8ae-4a8b-aa25-3c21e227c3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:49.606029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ea014-d8ae-4a8b-aa25-3c21e227c3d1 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:49.606504 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ea014-d8ae-4a8b-aa25-3c21e227c3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:49.606968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ea014-d8ae-4a8b-aa25-3c21e227c3d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:49.622650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ea014-d8ae-4a8b-aa25-3c21e227c3d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-df7ea014-d8ae-4a8b-aa25-3c21e227c3d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:49.627818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:49.628697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:49.629503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:49.629503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:49.629503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:49.629503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:49.629503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:49.629503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:49.629503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:49.629503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:49.629503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:49.629503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:49.629503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:49.629503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:49.633117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c17f2-c031-4bec-a4f8-d2e88fcb709d None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:49.633117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c17f2-c031-4bec-a4f8-d2e88fcb709d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:49.633300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c17f2-c031-4bec-a4f8-d2e88fcb709d None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:49.634778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 245/483] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:49.635206 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 239/484] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:32:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:49.639912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091a7ca-7580-4987-8da0-2ae0123c8edb None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:49.640205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091a7ca-7580-4987-8da0-2ae0123c8edb None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:49.640517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091a7ca-7580-4987-8da0-2ae0123c8edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:49.641871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ea014-d8ae-4a8b-aa25-3c21e227c3d1 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:49.694210 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:49.698750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-8091a7ca-7580-4987-8da0-2ae0123c8edb None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:49.700208 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8091a7ca-7580-4987-8da0-2ae0123c8edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:49.805865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ea014-d8ae-4a8b-aa25-3c21e227c3d1 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:49.824474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 246/485] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:49.826137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654cec9-c995-479f-9497-7e7a761fe282 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:49.826584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654cec9-c995-479f-9497-7e7a761fe282 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:49.827014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654cec9-c995-479f-9497-7e7a761fe282 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:49.828709 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 247/486] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:49.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27033c02-6d60-41fe-9698-27a56b10a079 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:49.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27033c02-6d60-41fe-9698-27a56b10a079 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:49.837718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27033c02-6d60-41fe-9698-27a56b10a079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:49.866872 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1f673-1e41-402b-8efc-763ff3d19a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:49.867517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1f673-1e41-402b-8efc-763ff3d19a82 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:49.867738 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1f673-1e41-402b-8efc-763ff3d19a82 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:49.868006 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1f673-1e41-402b-8efc-763ff3d19a82 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:49.868747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1f673-1e41-402b-8efc-763ff3d19a82 None admin] RBAC: Adding query filter params (name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:49.869030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1f673-1e41-402b-8efc-763ff3d19a82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:49.870411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1f673-1e41-402b-8efc-763ff3d19a82 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:49.871654 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-27033c02-6d60-41fe-9698-27a56b10a079 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:49.872846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-27033c02-6d60-41fe-9698-27a56b10a079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:49.887394 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 240/487] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:32:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:49.890473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:49.895002 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1f673-1e41-402b-8efc-763ff3d19a82 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:49.966182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 248/488] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:50.022363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:50.045532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1f673-1e41-402b-8efc-763ff3d19a82 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:50.078853 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785775d-896b-450f-8a3f-586f13af5fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:50.079474 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785775d-896b-450f-8a3f-586f13af5fd9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:50.079751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785775d-896b-450f-8a3f-586f13af5fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:50.080017 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785775d-896b-450f-8a3f-586f13af5fd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:50.090543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785775d-896b-450f-8a3f-586f13af5fd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:50.092047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785775d-896b-450f-8a3f-586f13af5fd9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-5785775d-896b-450f-8a3f-586f13af5fd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:50.106999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:50.107896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:50.108998 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:50.108998 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:50.108998 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:50.108998 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 241/489] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:32:50.114796 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27033c02-6d60-41fe-9698-27a56b10a079 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:50.181990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785775d-896b-450f-8a3f-586f13af5fd9 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:50.262183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27033c02-6d60-41fe-9698-27a56b10a079 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:50.324234 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e506fe1-af4c-47cb-b960-348a22d2225e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:50.324853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e506fe1-af4c-47cb-b960-348a22d2225e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:50.325136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e506fe1-af4c-47cb-b960-348a22d2225e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:50.325412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e506fe1-af4c-47cb-b960-348a22d2225e None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:50.326092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e506fe1-af4c-47cb-b960-348a22d2225e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:50.326645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e506fe1-af4c-47cb-b960-348a22d2225e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:50.327874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e506fe1-af4c-47cb-b960-348a22d2225e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:50.340056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 249/490] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:50.340918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e506fe1-af4c-47cb-b960-348a22d2225e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:50.347390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785775d-896b-450f-8a3f-586f13af5fd9 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:50.350577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:50.387848 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af1c67-73cf-4380-9dd6-e494bfa13663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:50.388202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af1c67-73cf-4380-9dd6-e494bfa13663 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:50.388480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af1c67-73cf-4380-9dd6-e494bfa13663 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:50.388748 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af1c67-73cf-4380-9dd6-e494bfa13663 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:50.391478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:50.402167 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af1c67-73cf-4380-9dd6-e494bfa13663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-f6af1c67-73cf-4380-9dd6-e494bfa13663 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:50.408569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:50.409663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:50.410610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 242/491] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:32:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:50.411310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af1c67-73cf-4380-9dd6-e494bfa13663 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:50.454066 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e506fe1-af4c-47cb-b960-348a22d2225e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:50.489991 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d18e7-175b-4bd9-b71e-568c9dd13fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:50.491396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d18e7-175b-4bd9-b71e-568c9dd13fc1 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:50.491822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d18e7-175b-4bd9-b71e-568c9dd13fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:50.492264 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d18e7-175b-4bd9-b71e-568c9dd13fc1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:50.503067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d18e7-175b-4bd9-b71e-568c9dd13fc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-a03d18e7-175b-4bd9-b71e-568c9dd13fc1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:50.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:50.509771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:50.510679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:50.510679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:50.510679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:50.510679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:50.510679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:50.510679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:50.510679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:50.510679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:50.510679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:50.510679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:50.510679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:50.510679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:50.512495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 250/492] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:32:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:50.513437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d18e7-175b-4bd9-b71e-568c9dd13fc1 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:50.535044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af1c67-73cf-4380-9dd6-e494bfa13663 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:50.575233 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8ad45-70be-4bd8-a8ca-47a9dfa02963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:50.575861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8ad45-70be-4bd8-a8ca-47a9dfa02963 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:50.576141 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8ad45-70be-4bd8-a8ca-47a9dfa02963 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:50.576639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8ad45-70be-4bd8-a8ca-47a9dfa02963 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:50.585488 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8ad45-70be-4bd8-a8ca-47a9dfa02963 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:50.586965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8ad45-70be-4bd8-a8ca-47a9dfa02963 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:50.587429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-2cf8ad45-70be-4bd8-a8ca-47a9dfa02963 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:32:50.601802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:50.602948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:50.603856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:50.603856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:50.603856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:32:50.603856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:32:50.603856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:32:50.603856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:32:50.603856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:32:50.603856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:32:50.603856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:32:50.603856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:50.603856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 243/493] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 08:32:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:50.604674 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8ad45-70be-4bd8-a8ca-47a9dfa02963 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:50.631183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d18e7-175b-4bd9-b71e-568c9dd13fc1 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:50.667606 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11beffd4-3862-4b89-b213-a3b8ebf61f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:50.668362 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11beffd4-3862-4b89-b213-a3b8ebf61f3b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:50.668771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11beffd4-3862-4b89-b213-a3b8ebf61f3b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:50.669187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11beffd4-3862-4b89-b213-a3b8ebf61f3b None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:50.670077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11beffd4-3862-4b89-b213-a3b8ebf61f3b None admin] RBAC: Adding query filter params (name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:50.670456 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11beffd4-3862-4b89-b213-a3b8ebf61f3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:50.671956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11beffd4-3862-4b89-b213-a3b8ebf61f3b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:50.672588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:50.683124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 251/494] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:32:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:50.684018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11beffd4-3862-4b89-b213-a3b8ebf61f3b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:50.725101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8ad45-70be-4bd8-a8ca-47a9dfa02963 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:50.768078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09dfd6-faed-44d2-9019-97ff97d6e68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:50.768913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09dfd6-faed-44d2-9019-97ff97d6e68f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:50.769331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09dfd6-faed-44d2-9019-97ff97d6e68f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:50.769801 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09dfd6-faed-44d2-9019-97ff97d6e68f None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:50.770598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09dfd6-faed-44d2-9019-97ff97d6e68f None admin] RBAC: Adding query filter params (name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:50.771039 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09dfd6-faed-44d2-9019-97ff97d6e68f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:50.773186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09dfd6-faed-44d2-9019-97ff97d6e68f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:50.782832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 244/495] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:32:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:50.783866 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09dfd6-faed-44d2-9019-97ff97d6e68f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:50.800453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11beffd4-3862-4b89-b213-a3b8ebf61f3b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:50.833058 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:50.844172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee96556-e1f7-488a-b3b2-3fc038a43407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:50.845055 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee96556-e1f7-488a-b3b2-3fc038a43407 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:50.845415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee96556-e1f7-488a-b3b2-3fc038a43407 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:50.845826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee96556-e1f7-488a-b3b2-3fc038a43407 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:50.846951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee96556-e1f7-488a-b3b2-3fc038a43407 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:50.847271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee96556-e1f7-488a-b3b2-3fc038a43407 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:50.848975 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee96556-e1f7-488a-b3b2-3fc038a43407 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:50.895957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09dfd6-faed-44d2-9019-97ff97d6e68f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:50.907697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 252/496] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 08:32:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:50.912263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee96556-e1f7-488a-b3b2-3fc038a43407 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:50.920207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:50.939012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae3b47-cbea-4e1d-a997-f7799f2c38e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:50.939639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae3b47-cbea-4e1d-a997-f7799f2c38e8 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:50.939924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae3b47-cbea-4e1d-a997-f7799f2c38e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:50.940200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae3b47-cbea-4e1d-a997-f7799f2c38e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:50.952420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae3b47-cbea-4e1d-a997-f7799f2c38e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:50.953924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae3b47-cbea-4e1d-a997-f7799f2c38e8 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-8cae3b47-cbea-4e1d-a997-f7799f2c38e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:50.970151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:50.971285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:50.972118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:50.972118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:50.972118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:50.972118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 245/497] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:50.972360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0237e55-8b8e-42dd-afdb-8365a8972da3 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:50.973450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0237e55-8b8e-42dd-afdb-8365a8972da3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:50.973450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0237e55-8b8e-42dd-afdb-8365a8972da3 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:50.974275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 246/498] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:50.978732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0237e55-8b8e-42dd-afdb-8365a8972da3 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:51.022845 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee96556-e1f7-488a-b3b2-3fc038a43407 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:51.055435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59702f16-2710-48c0-93eb-e2a882c4937b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:51.056055 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59702f16-2710-48c0-93eb-e2a882c4937b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.056337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59702f16-2710-48c0-93eb-e2a882c4937b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.056610 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59702f16-2710-48c0-93eb-e2a882c4937b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.066280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59702f16-2710-48c0-93eb-e2a882c4937b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:51.067737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59702f16-2710-48c0-93eb-e2a882c4937b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:51.077620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0237e55-8b8e-42dd-afdb-8365a8972da3 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-59702f16-2710-48c0-93eb-e2a882c4937b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:51.082310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:51.083339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:51.089500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:51.089500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:51.089500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:51.089500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 253/499] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:51.089500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59702f16-2710-48c0-93eb-e2a882c4937b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:51.115126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c3418-dfff-4b4b-b8a5-c02058a4d528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:51.115946 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c3418-dfff-4b4b-b8a5-c02058a4d528 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.116225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c3418-dfff-4b4b-b8a5-c02058a4d528 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.116495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c3418-dfff-4b4b-b8a5-c02058a4d528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.125216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:51.154512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:51.183213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c3418-dfff-4b4b-b8a5-c02058a4d528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:51.184050 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59702f16-2710-48c0-93eb-e2a882c4937b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:51.184681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c3418-dfff-4b4b-b8a5-c02058a4d528 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:51.218325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db121320-9465-4237-b4f6-a9893671b8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:51.218927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db121320-9465-4237-b4f6-a9893671b8ec None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.219210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db121320-9465-4237-b4f6-a9893671b8ec None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.219485 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db121320-9465-4237-b4f6-a9893671b8ec None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.220336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db121320-9465-4237-b4f6-a9893671b8ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:51.220859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db121320-9465-4237-b4f6-a9893671b8ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:51.223410 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db121320-9465-4237-b4f6-a9893671b8ec None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:51.239752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 254/500] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:51.241605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba2bcd-59a8-49b3-a8b2-a54812ddb537 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.242018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba2bcd-59a8-49b3-a8b2-a54812ddb537 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.242212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba2bcd-59a8-49b3-a8b2-a54812ddb537 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.245729 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 255/501] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:51.245729 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c135f5-2d43-4e6c-937b-a2e5fd6020cd None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.245729 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c135f5-2d43-4e6c-937b-a2e5fd6020cd None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.245729 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c135f5-2d43-4e6c-937b-a2e5fd6020cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.263239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-71c135f5-2d43-4e6c-937b-a2e5fd6020cd None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:51.264449 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-71c135f5-2d43-4e6c-937b-a2e5fd6020cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:51.288873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 247/502] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:51.289651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c3418-dfff-4b4b-b8a5-c02058a4d528 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:51.329874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 256/503] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:51.330559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c135f5-2d43-4e6c-937b-a2e5fd6020cd None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:51.363585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:51.370220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:51.406483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c3418-dfff-4b4b-b8a5-c02058a4d528 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:51.439013 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c135f5-2d43-4e6c-937b-a2e5fd6020cd None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:51.441818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db584c17-1a98-44f6-b2a0-0e39abf4e006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:51.442128 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db584c17-1a98-44f6-b2a0-0e39abf4e006 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.442401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db584c17-1a98-44f6-b2a0-0e39abf4e006 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.442668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db584c17-1a98-44f6-b2a0-0e39abf4e006 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.443294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db584c17-1a98-44f6-b2a0-0e39abf4e006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:51.443591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db584c17-1a98-44f6-b2a0-0e39abf4e006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:51.444989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db584c17-1a98-44f6-b2a0-0e39abf4e006 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:51.461904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 248/504] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:51.461904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b548b-699c-446a-973e-9227b6c2a714 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.461904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b548b-699c-446a-973e-9227b6c2a714 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.461904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b548b-699c-446a-973e-9227b6c2a714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.477107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b122db-4326-458f-b771-1e29431845a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:51.477766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b122db-4326-458f-b771-1e29431845a9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.478053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b122db-4326-458f-b771-1e29431845a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.478328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b122db-4326-458f-b771-1e29431845a9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.478438 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-528b548b-699c-446a-973e-9227b6c2a714 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:51.479774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-528b548b-699c-446a-973e-9227b6c2a714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:51.485985 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b122db-4326-458f-b771-1e29431845a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:51.487436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b122db-4326-458f-b771-1e29431845a9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-45b122db-4326-458f-b771-1e29431845a9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:32:51.500133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:51.500994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:51.501900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:51.501900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:51.501900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:32:51.501900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:32:51.501900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:32:51.501900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:32:51.501900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:32:51.501900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:32:51.501900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 08:32:51.501900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:51.505942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 257/505] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:51.506727 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b122db-4326-458f-b771-1e29431845a9 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:51.561920 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 249/506] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:51.565313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b548b-699c-446a-973e-9227b6c2a714 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:51.614168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:51.627395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b122db-4326-458f-b771-1e29431845a9 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:51.629450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:51.668202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03afe97-6b2a-4a1e-badf-b98a617880e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:51.668825 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03afe97-6b2a-4a1e-badf-b98a617880e7 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.669111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03afe97-6b2a-4a1e-badf-b98a617880e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.669384 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03afe97-6b2a-4a1e-badf-b98a617880e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.675972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b548b-699c-446a-973e-9227b6c2a714 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:51.680440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03afe97-6b2a-4a1e-badf-b98a617880e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:51.681923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03afe97-6b2a-4a1e-badf-b98a617880e7 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-e03afe97-6b2a-4a1e-badf-b98a617880e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:51.697071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:51.698018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:51.698865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:51.698865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:51.698865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:51.698865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 258/507] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:51.699566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03afe97-6b2a-4a1e-badf-b98a617880e7 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:51.716056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e6b20-7409-478d-802d-a09e1f3e3ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:51.716683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e6b20-7409-478d-802d-a09e1f3e3ee2 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.716958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e6b20-7409-478d-802d-a09e1f3e3ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.717231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e6b20-7409-478d-802d-a09e1f3e3ee2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.724269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e6b20-7409-478d-802d-a09e1f3e3ee2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:51.726285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e6b20-7409-478d-802d-a09e1f3e3ee2 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-0f1e6b20-7409-478d-802d-a09e1f3e3ee2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:32:51.737839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:51.738721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:51.739599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:51.739599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:51.739599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:32:51.739599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:32:51.739599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:32:51.739599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:32:51.739599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:32:51.739599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:32:51.739599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:32:51.739599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:51.739599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 250/508] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:51.740172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e6b20-7409-478d-802d-a09e1f3e3ee2 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:51.807136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03afe97-6b2a-4a1e-badf-b98a617880e7 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:51.841119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e6db1-ed43-42b1-962c-d004dcd5644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:51.841408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e6b20-7409-478d-802d-a09e1f3e3ee2 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:51.841645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e6db1-ed43-42b1-962c-d004dcd5644c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.841952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e6db1-ed43-42b1-962c-d004dcd5644c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.842227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e6db1-ed43-42b1-962c-d004dcd5644c None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.842944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e6db1-ed43-42b1-962c-d004dcd5644c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:51.843249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e6db1-ed43-42b1-962c-d004dcd5644c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:51.844695 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e6db1-ed43-42b1-962c-d004dcd5644c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:51.854812 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:51.862704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 259/509] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Oct 4 08:32:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:51.862838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:51.863533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e6db1-ed43-42b1-962c-d004dcd5644c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:51.883536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7025c-3869-4407-89b9-8a175c0cb18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:51.884161 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7025c-3869-4407-89b9-8a175c0cb18a None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.884438 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7025c-3869-4407-89b9-8a175c0cb18a None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.884710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7025c-3869-4407-89b9-8a175c0cb18a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.894578 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7025c-3869-4407-89b9-8a175c0cb18a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-d6e7025c-3869-4407-89b9-8a175c0cb18a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:51.900224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:51.901735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:51.902763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:51.902763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:51.902763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:51.902763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:51.902763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:51.902763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:51.902763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:51.902763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:51.902763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:51.902763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:51.902763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:51.902763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:51.906042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 251/510] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:32:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:51.906912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7025c-3869-4407-89b9-8a175c0cb18a None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:51.961857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e6db1-ed43-42b1-962c-d004dcd5644c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:51.996007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a254c2c-2e5a-40e7-bcc0-5a33732963f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:51.996608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a254c2c-2e5a-40e7-bcc0-5a33732963f1 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:51.996884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a254c2c-2e5a-40e7-bcc0-5a33732963f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:51.997152 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a254c2c-2e5a-40e7-bcc0-5a33732963f1 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:51.997814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a254c2c-2e5a-40e7-bcc0-5a33732963f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:51.998119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a254c2c-2e5a-40e7-bcc0-5a33732963f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:51.999529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a254c2c-2e5a-40e7-bcc0-5a33732963f1 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:52.004630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7025c-3869-4407-89b9-8a175c0cb18a None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:52.011645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 260/511] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:52.012440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a254c2c-2e5a-40e7-bcc0-5a33732963f1 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:52.038471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb932f-aec5-44c9-a1fe-b997ff2836be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:52.039072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb932f-aec5-44c9-a1fe-b997ff2836be None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:52.039352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb932f-aec5-44c9-a1fe-b997ff2836be None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:52.039627 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb932f-aec5-44c9-a1fe-b997ff2836be None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:52.040350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb932f-aec5-44c9-a1fe-b997ff2836be None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:52.040659 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb932f-aec5-44c9-a1fe-b997ff2836be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:52.042121 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb932f-aec5-44c9-a1fe-b997ff2836be None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:52.068222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:52.082332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 252/512] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Oct 4 08:32:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:52.082332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb932f-aec5-44c9-a1fe-b997ff2836be None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:52.113227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a254c2c-2e5a-40e7-bcc0-5a33732963f1 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:52.118066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:52.146039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17332274-00ef-485f-83a6-11dd75c6175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:52.146659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17332274-00ef-485f-83a6-11dd75c6175b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:52.146941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17332274-00ef-485f-83a6-11dd75c6175b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:52.147211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17332274-00ef-485f-83a6-11dd75c6175b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:52.157882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17332274-00ef-485f-83a6-11dd75c6175b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:52.159333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17332274-00ef-485f-83a6-11dd75c6175b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-17332274-00ef-485f-83a6-11dd75c6175b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:52.173102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:52.174006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:52.174846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:52.174846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:52.174846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:52.174846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 261/513] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:52.175082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17332274-00ef-485f-83a6-11dd75c6175b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:52.180865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb932f-aec5-44c9-a1fe-b997ff2836be None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:52.214944 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa29fd7-b86b-4f33-ae02-8c7115abe444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:52.215551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa29fd7-b86b-4f33-ae02-8c7115abe444 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:52.215832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa29fd7-b86b-4f33-ae02-8c7115abe444 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:52.216107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa29fd7-b86b-4f33-ae02-8c7115abe444 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:52.271675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17332274-00ef-485f-83a6-11dd75c6175b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:52.282055 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa29fd7-b86b-4f33-ae02-8c7115abe444 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:52.282173 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:52.283531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa29fd7-b86b-4f33-ae02-8c7115abe444 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:52.306705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446c90d-1deb-420b-91e2-e39a79e40c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:52.307299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446c90d-1deb-420b-91e2-e39a79e40c5b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:52.307580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446c90d-1deb-420b-91e2-e39a79e40c5b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:52.307860 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446c90d-1deb-420b-91e2-e39a79e40c5b None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:52.308515 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446c90d-1deb-420b-91e2-e39a79e40c5b None admin] RBAC: Adding query filter params (name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:52.308820 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446c90d-1deb-420b-91e2-e39a79e40c5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:52.310218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446c90d-1deb-420b-91e2-e39a79e40c5b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:52.317854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 262/514] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:32:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:52.318639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446c90d-1deb-420b-91e2-e39a79e40c5b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:52.323193 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:52.384578 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 253/515] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:52.385326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa29fd7-b86b-4f33-ae02-8c7115abe444 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:52.419499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446c90d-1deb-420b-91e2-e39a79e40c5b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:52.453390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a77a5-e0c9-4e10-a7bb-484b0ed20875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:52.454042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a77a5-e0c9-4e10-a7bb-484b0ed20875 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:52.454325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a77a5-e0c9-4e10-a7bb-484b0ed20875 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:52.454601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a77a5-e0c9-4e10-a7bb-484b0ed20875 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:52.461088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a77a5-e0c9-4e10-a7bb-484b0ed20875 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:52.462559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a77a5-e0c9-4e10-a7bb-484b0ed20875 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-021a77a5-e0c9-4e10-a7bb-484b0ed20875 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:32:52.473951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:52.474838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:52.476235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:32:52.476235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:32:52.476235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:32:52.476235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:32:52.476235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:32:52.476235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:32:52.476235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:32:52.476235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:52.476235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 263/516] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:52.476706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a77a5-e0c9-4e10-a7bb-484b0ed20875 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:52.482706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa29fd7-b86b-4f33-ae02-8c7115abe444 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:52.511216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:52.518678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a164e-6e8d-479b-8c27-04311a5be490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:52.519276 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a164e-6e8d-479b-8c27-04311a5be490 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:52.519553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a164e-6e8d-479b-8c27-04311a5be490 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:52.519832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a164e-6e8d-479b-8c27-04311a5be490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:52.527795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:52.580692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a77a5-e0c9-4e10-a7bb-484b0ed20875 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:52.585597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a164e-6e8d-479b-8c27-04311a5be490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:52.587095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a164e-6e8d-479b-8c27-04311a5be490 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:52.613982 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e10d9-ee6a-43cb-9421-5c7d33dc2c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:52.614578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e10d9-ee6a-43cb-9421-5c7d33dc2c86 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:52.614911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e10d9-ee6a-43cb-9421-5c7d33dc2c86 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:52.615192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e10d9-ee6a-43cb-9421-5c7d33dc2c86 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:52.615917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e10d9-ee6a-43cb-9421-5c7d33dc2c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:52.616220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e10d9-ee6a-43cb-9421-5c7d33dc2c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:52.617572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e10d9-ee6a-43cb-9421-5c7d33dc2c86 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:52.629666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 264/517] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:52.630472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e10d9-ee6a-43cb-9421-5c7d33dc2c86 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:52.694699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 254/518] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:52.695450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a164e-6e8d-479b-8c27-04311a5be490 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:52.725398 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:52.734559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e10d9-ee6a-43cb-9421-5c7d33dc2c86 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:52.737428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:52.768772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398adf50-e9ea-4193-afa7-940fa8c0d0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:52.769390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398adf50-e9ea-4193-afa7-940fa8c0d0d2 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:52.769672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398adf50-e9ea-4193-afa7-940fa8c0d0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:52.769980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398adf50-e9ea-4193-afa7-940fa8c0d0d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:52.780211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398adf50-e9ea-4193-afa7-940fa8c0d0d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:52.781678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398adf50-e9ea-4193-afa7-940fa8c0d0d2 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:52.795323 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a164e-6e8d-479b-8c27-04311a5be490 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-398adf50-e9ea-4193-afa7-940fa8c0d0d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:52.796263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:52.797297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:52.798203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:52.798203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:52.798203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:52.801882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 265/519] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:52.802652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398adf50-e9ea-4193-afa7-940fa8c0d0d2 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:52.832211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7df79-a43d-4dc6-aef9-799dccf95fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:52.832827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7df79-a43d-4dc6-aef9-799dccf95fb7 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:52.833116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7df79-a43d-4dc6-aef9-799dccf95fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:52.833387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7df79-a43d-4dc6-aef9-799dccf95fb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:52.843735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7df79-a43d-4dc6-aef9-799dccf95fb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:52.845266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb7df79-a43d-4dc6-aef9-799dccf95fb7 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-0cb7df79-a43d-4dc6-aef9-799dccf95fb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:52.860701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:52.861823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:52.867881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:52.867881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:52.867881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:52.867881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 255/520] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:52.867881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7df79-a43d-4dc6-aef9-799dccf95fb7 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:52.914298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398adf50-e9ea-4193-afa7-940fa8c0d0d2 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:52.947161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aa5c6-52af-4155-885a-ef7021c683cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:52.947777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa5c6-52af-4155-885a-ef7021c683cf None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:52.948042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa5c6-52af-4155-885a-ef7021c683cf None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:52.948314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa5c6-52af-4155-885a-ef7021c683cf None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:52.948995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aa5c6-52af-4155-885a-ef7021c683cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:52.949298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aa5c6-52af-4155-885a-ef7021c683cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:52.950777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0aa5c6-52af-4155-885a-ef7021c683cf None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:52.951249 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:52.961396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7df79-a43d-4dc6-aef9-799dccf95fb7 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:52.968723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 266/521] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:52.969528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aa5c6-52af-4155-885a-ef7021c683cf None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:52.974893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:52.995463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d863f9d-2e90-45dc-8d50-403b7e98ed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:52.996072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d863f9d-2e90-45dc-8d50-403b7e98ed86 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:52.996355 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d863f9d-2e90-45dc-8d50-403b7e98ed86 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:52.996624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d863f9d-2e90-45dc-8d50-403b7e98ed86 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:52.997271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d863f9d-2e90-45dc-8d50-403b7e98ed86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:52.998768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d863f9d-2e90-45dc-8d50-403b7e98ed86 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:53.019763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 256/522] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:32:52 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:32:53.020749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d863f9d-2e90-45dc-8d50-403b7e98ed86 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:53.078596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0aa5c6-52af-4155-885a-ef7021c683cf None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:53.115934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c4d67-68cd-425d-b91c-e733e07cd6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:53.116670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c4d67-68cd-425d-b91c-e733e07cd6a0 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:53.117021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c4d67-68cd-425d-b91c-e733e07cd6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:53.117365 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c4d67-68cd-425d-b91c-e733e07cd6a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:53.128411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c4d67-68cd-425d-b91c-e733e07cd6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:53.130020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c4d67-68cd-425d-b91c-e733e07cd6a0 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:53.144957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d863f9d-2e90-45dc-8d50-403b7e98ed86 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-d97c4d67-68cd-425d-b91c-e733e07cd6a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:53.146197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:53.147366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:53.148522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:53.148522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:53.148522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:53.148522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 267/523] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:53.153451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c4d67-68cd-425d-b91c-e733e07cd6a0 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:53.186236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a3216-f592-42cb-8b8c-1c73040fdbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:53.186937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a3216-f592-42cb-8b8c-1c73040fdbf6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:53.187227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a3216-f592-42cb-8b8c-1c73040fdbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:53.187503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a3216-f592-42cb-8b8c-1c73040fdbf6 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:53.188441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a3216-f592-42cb-8b8c-1c73040fdbf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:53.188441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a3216-f592-42cb-8b8c-1c73040fdbf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:53.190182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a3216-f592-42cb-8b8c-1c73040fdbf6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:53.203125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 257/524] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:53.204041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a3216-f592-42cb-8b8c-1c73040fdbf6 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:53.212992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:53.223421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:53.275413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c4d67-68cd-425d-b91c-e733e07cd6a0 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:53.311613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5b5c5-0e7f-4333-bc32-54395154ee68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:53.312337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5b5c5-0e7f-4333-bc32-54395154ee68 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:53.312621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5b5c5-0e7f-4333-bc32-54395154ee68 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:53.312898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5b5c5-0e7f-4333-bc32-54395154ee68 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:53.313542 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5b5c5-0e7f-4333-bc32-54395154ee68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:53.313877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5b5c5-0e7f-4333-bc32-54395154ee68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:53.315254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5b5c5-0e7f-4333-bc32-54395154ee68 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:53.321515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a3216-f592-42cb-8b8c-1c73040fdbf6 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:53.331262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 268/525] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:53.331262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5b5c5-0e7f-4333-bc32-54395154ee68 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:53.359770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230d0a9-5f01-4d4d-b44d-3e764169a590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:53.360377 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230d0a9-5f01-4d4d-b44d-3e764169a590 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:53.360658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230d0a9-5f01-4d4d-b44d-3e764169a590 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:53.360939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230d0a9-5f01-4d4d-b44d-3e764169a590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:53.361668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230d0a9-5f01-4d4d-b44d-3e764169a590 None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=ee8a19f4d9704408a8b737095e5885f1) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:53.362015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230d0a9-5f01-4d4d-b44d-3e764169a590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:53.363456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230d0a9-5f01-4d4d-b44d-3e764169a590 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:53.368833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 258/526] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:53 2022] GET /identity/v3/role_assignments?user.id=ee8a19f4d9704408a8b737095e5885f1&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:53.369621 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230d0a9-5f01-4d4d-b44d-3e764169a590 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:53.452818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5b5c5-0e7f-4333-bc32-54395154ee68 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:53.464738 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:53.480836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:53.508778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f1059-9155-4647-8349-4edce7b8cc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:53.510474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f1059-9155-4647-8349-4edce7b8cc60 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:53.510992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f1059-9155-4647-8349-4edce7b8cc60 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:53.511464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f1059-9155-4647-8349-4edce7b8cc60 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:53.512082 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230d0a9-5f01-4d4d-b44d-3e764169a590 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:53.512624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f1059-9155-4647-8349-4edce7b8cc60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:53.513164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f1059-9155-4647-8349-4edce7b8cc60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:53.516304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f1059-9155-4647-8349-4edce7b8cc60 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:53.537149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 269/527] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:53.538516 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f1059-9155-4647-8349-4edce7b8cc60 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:53.563527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186c507-422b-4407-a547-099872df8db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:53.564169 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186c507-422b-4407-a547-099872df8db5 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:53.564442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186c507-422b-4407-a547-099872df8db5 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:53.564720 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186c507-422b-4407-a547-099872df8db5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:53.574277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186c507-422b-4407-a547-099872df8db5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:53.576688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186c507-422b-4407-a547-099872df8db5 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-2186c507-422b-4407-a547-099872df8db5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:32:53.593288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:53.594432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:53.595335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:53.595335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:53.595335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:32:53.595335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:32:53.595335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:32:53.595335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:32:53.595335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:32:53.595335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:32:53.595335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 08:32:53.595335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:53.595335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 259/528] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:32:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:53.595335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186c507-422b-4407-a547-099872df8db5 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:53.676459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f1059-9155-4647-8349-4edce7b8cc60 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:53.710875 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186c507-422b-4407-a547-099872df8db5 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:53.712349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ae281-3b42-4e31-967f-7258c555e502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:53.713033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ae281-3b42-4e31-967f-7258c555e502 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:53.713393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ae281-3b42-4e31-967f-7258c555e502 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:53.713848 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ae281-3b42-4e31-967f-7258c555e502 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:53.721910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ae281-3b42-4e31-967f-7258c555e502 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:53.723586 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ae281-3b42-4e31-967f-7258c555e502 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-490ae281-3b42-4e31-967f-7258c555e502 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:32:53.735628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:53.736538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:53.737395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:32:53.737395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:32:53.737395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:32:53.737395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:32:53.737395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:32:53.737395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:32:53.737395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:32:53.737395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:53.742005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 270/529] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:53.742933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ae281-3b42-4e31-967f-7258c555e502 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:53.746836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:53.754521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fe1b2-e4e8-4b03-942a-b28aabc24eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:53.755426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fe1b2-e4e8-4b03-942a-b28aabc24eef None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:53.755880 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fe1b2-e4e8-4b03-942a-b28aabc24eef None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:53.756387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963fe1b2-e4e8-4b03-942a-b28aabc24eef None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:53.764334 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fe1b2-e4e8-4b03-942a-b28aabc24eef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:53.766232 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963fe1b2-e4e8-4b03-942a-b28aabc24eef None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-963fe1b2-e4e8-4b03-942a-b28aabc24eef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:32:53.779579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:53.780449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:53.781296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:32:53.781296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:32:53.781296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:32:53.781296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:32:53.781296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:32:53.781296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:32:53.781296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:32:53.781296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:53.781722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:53.788823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 260/530] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:53.789991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fe1b2-e4e8-4b03-942a-b28aabc24eef None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:53.851405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ae281-3b42-4e31-967f-7258c555e502 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:53.888133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd98025-8c62-4f8c-bdae-5908f9f030f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:53.888825 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd98025-8c62-4f8c-bdae-5908f9f030f2 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:53.889175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd98025-8c62-4f8c-bdae-5908f9f030f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:53.889514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd98025-8c62-4f8c-bdae-5908f9f030f2 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:53.890343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd98025-8c62-4f8c-bdae-5908f9f030f2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:53.890737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd98025-8c62-4f8c-bdae-5908f9f030f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:53.892239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd98025-8c62-4f8c-bdae-5908f9f030f2 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:53.901730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963fe1b2-e4e8-4b03-942a-b28aabc24eef None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:53.910362 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 271/531] 10.222.0.26 () {56 vars in 1139 bytes} [Tue Oct 4 08:32:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:53.911343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd98025-8c62-4f8c-bdae-5908f9f030f2 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:53.938224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de114dd-3662-4691-a241-70963f82cd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:53.939220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de114dd-3662-4691-a241-70963f82cd8c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:53.939703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de114dd-3662-4691-a241-70963f82cd8c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:53.940253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de114dd-3662-4691-a241-70963f82cd8c None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:53.941166 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de114dd-3662-4691-a241-70963f82cd8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:53.941747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de114dd-3662-4691-a241-70963f82cd8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:53.943452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de114dd-3662-4691-a241-70963f82cd8c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:53.964283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 261/532] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:32:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:53.965954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de114dd-3662-4691-a241-70963f82cd8c None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:54.012900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:54.019700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:54.021033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd98025-8c62-4f8c-bdae-5908f9f030f2 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:54.057627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2a0c8-ef06-447b-949a-cef33744ac20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:54.058299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2a0c8-ef06-447b-949a-cef33744ac20 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:54.058582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2a0c8-ef06-447b-949a-cef33744ac20 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:54.058900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2a0c8-ef06-447b-949a-cef33744ac20 None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:54.059535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2a0c8-ef06-447b-949a-cef33744ac20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:54.059842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2a0c8-ef06-447b-949a-cef33744ac20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:54.061296 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2a0c8-ef06-447b-949a-cef33744ac20 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:54.077934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 272/533] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:32:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:54.077934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2a0c8-ef06-447b-949a-cef33744ac20 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:54.081588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de114dd-3662-4691-a241-70963f82cd8c None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:54.119170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96e10b-eeb4-48d3-a7a2-2f15404ac442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:54.120126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96e10b-eeb4-48d3-a7a2-2f15404ac442 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:54.120622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96e10b-eeb4-48d3-a7a2-2f15404ac442 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:54.121091 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96e10b-eeb4-48d3-a7a2-2f15404ac442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:54.184106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2a0c8-ef06-447b-949a-cef33744ac20 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:54.192554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96e10b-eeb4-48d3-a7a2-2f15404ac442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:54.194395 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96e10b-eeb4-48d3-a7a2-2f15404ac442 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:54.221816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeafbff-9387-4ad9-9ed7-a25ccd313ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:54.222346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeafbff-9387-4ad9-9ed7-a25ccd313ede None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:54.222734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeafbff-9387-4ad9-9ed7-a25ccd313ede None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:54.223011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeafbff-9387-4ad9-9ed7-a25ccd313ede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:54.223738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeafbff-9387-4ad9-9ed7-a25ccd313ede None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=913a866ddc074b7f9aa142c70da98a69) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:54.224048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeafbff-9387-4ad9-9ed7-a25ccd313ede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:54.226203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeafbff-9387-4ad9-9ed7-a25ccd313ede None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:54.238026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 273/534] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:54 2022] GET /identity/v3/role_assignments?user.id=913a866ddc074b7f9aa142c70da98a69&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:54.239110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeafbff-9387-4ad9-9ed7-a25ccd313ede None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:54.250148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:54.276051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:54.361715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 262/535] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:54.367957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96e10b-eeb4-48d3-a7a2-2f15404ac442 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:54.453035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeafbff-9387-4ad9-9ed7-a25ccd313ede None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:54.492279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5b478-da1a-4d86-9b71-783ba2c9e75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:54.493200 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5b478-da1a-4d86-9b71-783ba2c9e75e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:54.493622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5b478-da1a-4d86-9b71-783ba2c9e75e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:54.494060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5b478-da1a-4d86-9b71-783ba2c9e75e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:54.495680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5b478-da1a-4d86-9b71-783ba2c9e75e None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=6f224ed840244faea117fe97c29c26d1) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:54.496224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5b478-da1a-4d86-9b71-783ba2c9e75e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:54.497835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5b478-da1a-4d86-9b71-783ba2c9e75e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:54.504209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 274/536] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:54 2022] GET /identity/v3/role_assignments?user.id=6f224ed840244faea117fe97c29c26d1&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:54.530336 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96e10b-eeb4-48d3-a7a2-2f15404ac442 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:54.581381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df424ce9-e9ba-4acb-88b6-58ea931e0b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:54.582054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df424ce9-e9ba-4acb-88b6-58ea931e0b45 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:54.582335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df424ce9-e9ba-4acb-88b6-58ea931e0b45 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:54.582608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df424ce9-e9ba-4acb-88b6-58ea931e0b45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:54.593671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df424ce9-e9ba-4acb-88b6-58ea931e0b45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:54.595164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df424ce9-e9ba-4acb-88b6-58ea931e0b45 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-df424ce9-e9ba-4acb-88b6-58ea931e0b45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:54.617656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:54.618834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:54.619972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:54.619972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:54.619972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:54.623310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 263/537] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:32:54.628791 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5b478-da1a-4d86-9b71-783ba2c9e75e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:54.763733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:54.787242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5b478-da1a-4d86-9b71-783ba2c9e75e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:54.831530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09888df-5bd5-4d35-aecc-1df9c6926d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:54.832160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09888df-5bd5-4d35-aecc-1df9c6926d38 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:54.832447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09888df-5bd5-4d35-aecc-1df9c6926d38 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:54.832724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09888df-5bd5-4d35-aecc-1df9c6926d38 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:54.833382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09888df-5bd5-4d35-aecc-1df9c6926d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:54.833709 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09888df-5bd5-4d35-aecc-1df9c6926d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:54.835198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09888df-5bd5-4d35-aecc-1df9c6926d38 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:54.860431 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 275/538] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:54.867149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df424ce9-e9ba-4acb-88b6-58ea931e0b45 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:54.870224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:55.009524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df424ce9-e9ba-4acb-88b6-58ea931e0b45 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:55.062020 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a19e9f-decd-490f-8305-d1dcce9bf98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:55.062654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a19e9f-decd-490f-8305-d1dcce9bf98c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:55.062933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a19e9f-decd-490f-8305-d1dcce9bf98c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:55.063205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a19e9f-decd-490f-8305-d1dcce9bf98c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:55.076251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a19e9f-decd-490f-8305-d1dcce9bf98c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:55.077698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a19e9f-decd-490f-8305-d1dcce9bf98c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-e1a19e9f-decd-490f-8305-d1dcce9bf98c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:32:55.099036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:55.099935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:55.101379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:32:55.101379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:32:55.101379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:32:55.101379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:32:55.101379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:32:55.101379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:32:55.101379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:32:55.101379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:55.101379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 264/539] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:32:55.108277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09888df-5bd5-4d35-aecc-1df9c6926d38 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:55.273114 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09888df-5bd5-4d35-aecc-1df9c6926d38 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:55.310517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da4d12-41ac-4806-8722-891096fd3218 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:55.311169 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da4d12-41ac-4806-8722-891096fd3218 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:55.311702 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da4d12-41ac-4806-8722-891096fd3218 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:55.313508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 265/540] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:32:55.318628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a48e4-66c8-4675-a792-7b7fff8a2a18 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:55.319666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a48e4-66c8-4675-a792-7b7fff8a2a18 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:55.320301 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a48e4-66c8-4675-a792-7b7fff8a2a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:55.321416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0579ef-7366-4f87-b5ca-680fcffd4ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:55.322054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0579ef-7366-4f87-b5ca-680fcffd4ffe None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:55.322337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0579ef-7366-4f87-b5ca-680fcffd4ffe None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:55.322614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0579ef-7366-4f87-b5ca-680fcffd4ffe None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:55.323312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0579ef-7366-4f87-b5ca-680fcffd4ffe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:55.323622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0579ef-7366-4f87-b5ca-680fcffd4ffe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:55.325189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0579ef-7366-4f87-b5ca-680fcffd4ffe None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:55.340752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-e43a48e4-66c8-4675-a792-7b7fff8a2a18 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:55.342277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e43a48e4-66c8-4675-a792-7b7fff8a2a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:55.342937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 276/541] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:32:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:55.350856 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0579ef-7366-4f87-b5ca-680fcffd4ffe None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:55.356362 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:55.448239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:55.456115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 266/542] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:55.532539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0579ef-7366-4f87-b5ca-680fcffd4ffe None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:55.571793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1277c-9f86-4b94-9e2e-f0ecddea281c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:55.572724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1277c-9f86-4b94-9e2e-f0ecddea281c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:55.573230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1277c-9f86-4b94-9e2e-f0ecddea281c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:55.573718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1277c-9f86-4b94-9e2e-f0ecddea281c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:55.574696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1277c-9f86-4b94-9e2e-f0ecddea281c None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=50d3b75cbc5c4d1686f4d93248e04746) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:55.575220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1277c-9f86-4b94-9e2e-f0ecddea281c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:55.576835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1277c-9f86-4b94-9e2e-f0ecddea281c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:55.582838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 277/543] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:55 2022] GET /identity/v3/role_assignments?user.id=50d3b75cbc5c4d1686f4d93248e04746&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:55.801382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15cfbb-0036-40fc-9c82-312b012e4a1d None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:55.801790 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15cfbb-0036-40fc-9c82-312b012e4a1d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:55.801999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15cfbb-0036-40fc-9c82-312b012e4a1d None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:55.804934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 267/544] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:55.811017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0206c9-dd1a-43d8-b986-0187f4f9aa31 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:55.811017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0206c9-dd1a-43d8-b986-0187f4f9aa31 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:55.811017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0206c9-dd1a-43d8-b986-0187f4f9aa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:55.831010 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-bc0206c9-dd1a-43d8-b986-0187f4f9aa31 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:55.832484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-bc0206c9-dd1a-43d8-b986-0187f4f9aa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:55.872429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15cfbb-0036-40fc-9c82-312b012e4a1d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:55.911244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:55.927677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 278/545] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:32:56.022277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15cfbb-0036-40fc-9c82-312b012e4a1d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:56.075274 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333dd7d-9f0d-445a-aae7-5a2513ea40c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:56.075913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333dd7d-9f0d-445a-aae7-5a2513ea40c4 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:56.076185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333dd7d-9f0d-445a-aae7-5a2513ea40c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:56.076448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333dd7d-9f0d-445a-aae7-5a2513ea40c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:56.087876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333dd7d-9f0d-445a-aae7-5a2513ea40c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-1333dd7d-9f0d-445a-aae7-5a2513ea40c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:56.092647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:56.093523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:56.094743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 268/546] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:32:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:56.102814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0206c9-dd1a-43d8-b986-0187f4f9aa31 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:56.245036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0206c9-dd1a-43d8-b986-0187f4f9aa31 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:56.281190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e645d7a-2dd5-4c36-8080-24ef4d132def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:56.281817 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e645d7a-2dd5-4c36-8080-24ef4d132def None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:56.282100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e645d7a-2dd5-4c36-8080-24ef4d132def None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:56.282373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e645d7a-2dd5-4c36-8080-24ef4d132def None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:56.283037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e645d7a-2dd5-4c36-8080-24ef4d132def None admin] RBAC: Adding query filter params (name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:56.283340 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e645d7a-2dd5-4c36-8080-24ef4d132def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:56.284720 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e645d7a-2dd5-4c36-8080-24ef4d132def None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:56.293088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 279/547] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:32:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:56.297557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333dd7d-9f0d-445a-aae7-5a2513ea40c4 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:56.299757 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e645d7a-2dd5-4c36-8080-24ef4d132def None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:56.362932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:56.401447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:56.423819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333dd7d-9f0d-445a-aae7-5a2513ea40c4 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:56.437037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e645d7a-2dd5-4c36-8080-24ef4d132def None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:56.462204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe4f9-8824-4f57-9062-034b409772e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:56.462836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe4f9-8824-4f57-9062-034b409772e8 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:56.463115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe4f9-8824-4f57-9062-034b409772e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:56.463384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe4f9-8824-4f57-9062-034b409772e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:56.475099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fe4f9-8824-4f57-9062-034b409772e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-026fe4f9-8824-4f57-9062-034b409772e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:56.480682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:56.481554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:56.482773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 269/548] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:32:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:56.487616 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe4f9-8824-4f57-9062-034b409772e8 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:56.517835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2caa08-5b48-43f8-93a8-3325f871504c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:56.518562 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2caa08-5b48-43f8-93a8-3325f871504c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:56.518842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2caa08-5b48-43f8-93a8-3325f871504c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:56.519171 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2caa08-5b48-43f8-93a8-3325f871504c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:56.540385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2caa08-5b48-43f8-93a8-3325f871504c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:56.542002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2caa08-5b48-43f8-93a8-3325f871504c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-2b2caa08-5b48-43f8-93a8-3325f871504c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:56.567388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:56.569210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:56.570282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:56.570282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:56.570282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:56.570282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 280/549] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:32:56.576279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2caa08-5b48-43f8-93a8-3325f871504c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:56.596233 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe4f9-8824-4f57-9062-034b409772e8 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:56.630131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:56.673886 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be522a-008b-4b2b-b8bc-e5c81a49ff08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:56.674511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be522a-008b-4b2b-b8bc-e5c81a49ff08 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:56.674791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be522a-008b-4b2b-b8bc-e5c81a49ff08 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:56.675066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be522a-008b-4b2b-b8bc-e5c81a49ff08 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:56.675737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be522a-008b-4b2b-b8bc-e5c81a49ff08 None admin] RBAC: Adding query filter params (name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:56.676040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be522a-008b-4b2b-b8bc-e5c81a49ff08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:56.677433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be522a-008b-4b2b-b8bc-e5c81a49ff08 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:56.700479 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 270/550] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:32:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:56.701410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be522a-008b-4b2b-b8bc-e5c81a49ff08 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:56.794577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2caa08-5b48-43f8-93a8-3325f871504c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:56.813412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:56.830541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be522a-008b-4b2b-b8bc-e5c81a49ff08 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:56.871007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93580d22-e7e4-4537-9cfc-025cf9903587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:56.871650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93580d22-e7e4-4537-9cfc-025cf9903587 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:56.871932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93580d22-e7e4-4537-9cfc-025cf9903587 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:56.872208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93580d22-e7e4-4537-9cfc-025cf9903587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:56.884274 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93580d22-e7e4-4537-9cfc-025cf9903587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:56.885660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93580d22-e7e4-4537-9cfc-025cf9903587 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:56.894921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4221bf-da5a-41fd-8f36-52df91f08af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:56.895790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4221bf-da5a-41fd-8f36-52df91f08af4 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:56.896273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4221bf-da5a-41fd-8f36-52df91f08af4 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:56.896748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4221bf-da5a-41fd-8f36-52df91f08af4 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:56.898238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4221bf-da5a-41fd-8f36-52df91f08af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:56.898910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4221bf-da5a-41fd-8f36-52df91f08af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:56.900483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4221bf-da5a-41fd-8f36-52df91f08af4 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-93580d22-e7e4-4537-9cfc-025cf9903587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:56.900805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:56.901829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:56.904074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:56.904074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:56.904074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:56.904074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 271/551] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:56.909712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93580d22-e7e4-4537-9cfc-025cf9903587 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:56.930596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 281/552] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:32:56.944458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4221bf-da5a-41fd-8f36-52df91f08af4 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:56.953229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:57.036433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93580d22-e7e4-4537-9cfc-025cf9903587 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:57.070832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4221bf-da5a-41fd-8f36-52df91f08af4 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:57.074620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232c782-2e90-4ee4-888d-d5ef20da5125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:57.075351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232c782-2e90-4ee4-888d-d5ef20da5125 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.075701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232c782-2e90-4ee4-888d-d5ef20da5125 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.076047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232c782-2e90-4ee4-888d-d5ef20da5125 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.076765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232c782-2e90-4ee4-888d-d5ef20da5125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:57.077139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232c782-2e90-4ee4-888d-d5ef20da5125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:57.078822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232c782-2e90-4ee4-888d-d5ef20da5125 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:57.093828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 272/553] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:57.095334 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc4785-ad81-4bee-9368-7e92d588db4f None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.095608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc4785-ad81-4bee-9368-7e92d588db4f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.095884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc4785-ad81-4bee-9368-7e92d588db4f None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.101769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 273/554] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:57.102537 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc4785-ad81-4bee-9368-7e92d588db4f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:57.115387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4df93-4ed0-42ae-b8ae-745d174f3161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:57.116485 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4df93-4ed0-42ae-b8ae-745d174f3161 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.117081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4df93-4ed0-42ae-b8ae-745d174f3161 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.117554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4df93-4ed0-42ae-b8ae-745d174f3161 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.125912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4df93-4ed0-42ae-b8ae-745d174f3161 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:57.127788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4df93-4ed0-42ae-b8ae-745d174f3161 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-95f4df93-4ed0-42ae-b8ae-745d174f3161 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:32:57.141544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:57.142581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:57.143566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:57.143566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:57.143566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:32:57.143566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:32:57.143566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:32:57.143566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:32:57.143566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:32:57.143566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:32:57.143566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:32:57.143566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:57.149819 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 282/555] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:32:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:57.151400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2d577-1079-46a6-8d25-9ed10d4c3517 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.151999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2d577-1079-46a6-8d25-9ed10d4c3517 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.152475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2d577-1079-46a6-8d25-9ed10d4c3517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.172961 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-18a2d577-1079-46a6-8d25-9ed10d4c3517 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:57.174680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-18a2d577-1079-46a6-8d25-9ed10d4c3517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:57.200217 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:57.215451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:57.223311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc4785-ad81-4bee-9368-7e92d588db4f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:57.264257 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725102a-6140-49f7-8361-fad5a66b6ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:57.264812 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725102a-6140-49f7-8361-fad5a66b6ff9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.265091 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725102a-6140-49f7-8361-fad5a66b6ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.265366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725102a-6140-49f7-8361-fad5a66b6ff9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.273868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725102a-6140-49f7-8361-fad5a66b6ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:57.275596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f725102a-6140-49f7-8361-fad5a66b6ff9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:57.277507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 283/556] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:57.279215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2d577-1079-46a6-8d25-9ed10d4c3517 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-f725102a-6140-49f7-8361-fad5a66b6ff9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:32:57.302770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:57.305147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:57.306464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:57.306464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:57.306464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:32:57.306464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:32:57.306464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:32:57.306464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:32:57.306464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:32:57.306464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:32:57.306464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:32:57.306464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:57.306464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 274/557] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 08:32:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:57.310756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725102a-6140-49f7-8361-fad5a66b6ff9 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:57.402510 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2d577-1079-46a6-8d25-9ed10d4c3517 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:57.431841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725102a-6140-49f7-8361-fad5a66b6ff9 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:57.436235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26335c-03f0-40b5-8df2-1fd6090f5534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:57.436963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26335c-03f0-40b5-8df2-1fd6090f5534 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.437308 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26335c-03f0-40b5-8df2-1fd6090f5534 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.437669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26335c-03f0-40b5-8df2-1fd6090f5534 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.438507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26335c-03f0-40b5-8df2-1fd6090f5534 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:57.438880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26335c-03f0-40b5-8df2-1fd6090f5534 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:57.440372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26335c-03f0-40b5-8df2-1fd6090f5534 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:57.462757 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 284/558] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Oct 4 08:32:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:57.464218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192aab6-623b-43aa-bad7-636ff24e17f1 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.464572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192aab6-623b-43aa-bad7-636ff24e17f1 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.464950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192aab6-623b-43aa-bad7-636ff24e17f1 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.466498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 285/559] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:57.467314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192aab6-623b-43aa-bad7-636ff24e17f1 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:57.476682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cafc01-d71c-400e-9b3a-95215577d7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:57.477531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cafc01-d71c-400e-9b3a-95215577d7d1 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.478027 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cafc01-d71c-400e-9b3a-95215577d7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.478810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cafc01-d71c-400e-9b3a-95215577d7d1 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.480467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cafc01-d71c-400e-9b3a-95215577d7d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:57.481315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cafc01-d71c-400e-9b3a-95215577d7d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:57.483620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cafc01-d71c-400e-9b3a-95215577d7d1 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:57.507353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 275/560] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 08:32:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:57.508950 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1e7ae-3c66-44e4-97ca-03d82ef88e46 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.510034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:57.511172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1e7ae-3c66-44e4-97ca-03d82ef88e46 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.511801 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1e7ae-3c66-44e4-97ca-03d82ef88e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.526100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:57.541217 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-3dc1e7ae-3c66-44e4-97ca-03d82ef88e46 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:57.542353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3dc1e7ae-3c66-44e4-97ca-03d82ef88e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:57.593623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192aab6-623b-43aa-bad7-636ff24e17f1 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:57.621712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 276/561] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:57.621712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1e7ae-3c66-44e4-97ca-03d82ef88e46 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:57.628418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0c4fc-61e8-4595-b1e9-4cc7f2b649b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:57.629032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0c4fc-61e8-4595-b1e9-4cc7f2b649b2 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.629314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0c4fc-61e8-4595-b1e9-4cc7f2b649b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.629592 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0c4fc-61e8-4595-b1e9-4cc7f2b649b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.707626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0c4fc-61e8-4595-b1e9-4cc7f2b649b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:57.709292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0c4fc-61e8-4595-b1e9-4cc7f2b649b2 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:57.729549 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:57.751129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1e7ae-3c66-44e4-97ca-03d82ef88e46 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:57.788468 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d8768-f99e-4e54-a68b-de407b0c3dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:57.789160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d8768-f99e-4e54-a68b-de407b0c3dc7 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.789551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d8768-f99e-4e54-a68b-de407b0c3dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.789876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d8768-f99e-4e54-a68b-de407b0c3dc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.791969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:57.803921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d8768-f99e-4e54-a68b-de407b0c3dc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:57.805379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d8768-f99e-4e54-a68b-de407b0c3dc7 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-125d8768-f99e-4e54-a68b-de407b0c3dc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:57.822845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:57.823808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:57.824752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:57.824752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:57.824752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:57.828540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 277/562] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:57.829669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d8768-f99e-4e54-a68b-de407b0c3dc7 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:57.842405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 286/563] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:32:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:32:57.844038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0c4fc-61e8-4595-b1e9-4cc7f2b649b2 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:57.945618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d8768-f99e-4e54-a68b-de407b0c3dc7 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:57.958779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0c4fc-61e8-4595-b1e9-4cc7f2b649b2 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:57.976844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:57.979965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f5461-d8c2-48fd-9202-0f3565c5caef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:57.980597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f5461-d8c2-48fd-9202-0f3565c5caef None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.980890 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f5461-d8c2-48fd-9202-0f3565c5caef None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.981150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f5461-d8c2-48fd-9202-0f3565c5caef None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.992526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f5461-d8c2-48fd-9202-0f3565c5caef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:57.995967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb878dcb-c46e-40b3-b697-3b06c6f7ff23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:57.996563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb878dcb-c46e-40b3-b697-3b06c6f7ff23 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:57.996845 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb878dcb-c46e-40b3-b697-3b06c6f7ff23 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:57.997119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb878dcb-c46e-40b3-b697-3b06c6f7ff23 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:57.997796 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb878dcb-c46e-40b3-b697-3b06c6f7ff23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:57.998108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb878dcb-c46e-40b3-b697-3b06c6f7ff23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-f02f5461-d8c2-48fd-9202-0f3565c5caef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:57.998446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:57.999350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:58.002623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:58.002623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:58.002623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:58.002623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:58.002623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:58.002623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:58.002623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:58.002623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:58.002623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:58.002623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:58.002623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:58.002623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:58.003270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb878dcb-c46e-40b3-b697-3b06c6f7ff23 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:58.003349 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 278/564] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:32:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:58.003349 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f5461-d8c2-48fd-9202-0f3565c5caef None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:58.021230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 287/565] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:58.021230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb878dcb-c46e-40b3-b697-3b06c6f7ff23 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:58.063017 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:58.121092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f5461-d8c2-48fd-9202-0f3565c5caef None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:58.123534 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb878dcb-c46e-40b3-b697-3b06c6f7ff23 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:58.155399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7513f86-9290-4adb-a19a-42cf48a4d0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:58.156021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7513f86-9290-4adb-a19a-42cf48a4d0c3 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.156299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7513f86-9290-4adb-a19a-42cf48a4d0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.156514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de7dd1-a471-4bbe-b28a-9c0b32bba868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:58.156625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7513f86-9290-4adb-a19a-42cf48a4d0c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.157139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de7dd1-a471-4bbe-b28a-9c0b32bba868 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.157421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de7dd1-a471-4bbe-b28a-9c0b32bba868 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.157709 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de7dd1-a471-4bbe-b28a-9c0b32bba868 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.167631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7513f86-9290-4adb-a19a-42cf48a4d0c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:58.168197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de7dd1-a471-4bbe-b28a-9c0b32bba868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:58.169067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7513f86-9290-4adb-a19a-42cf48a4d0c3 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-a1de7dd1-a471-4bbe-b28a-9c0b32bba868 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:58.174048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:58.174954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:58.175804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 288/566] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:32:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:58.177107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de7dd1-a471-4bbe-b28a-9c0b32bba868 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-c7513f86-9290-4adb-a19a-42cf48a4d0c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:58.185366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:58.186362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:58.187243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:58.187243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:58.187243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:58.187243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 279/567] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:58.187931 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7513f86-9290-4adb-a19a-42cf48a4d0c3 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:58.215511 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:58.276382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de7dd1-a471-4bbe-b28a-9c0b32bba868 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:58.281666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:58.285442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7513f86-9290-4adb-a19a-42cf48a4d0c3 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:58.308787 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe6f4d-af02-4753-82f8-a99648737c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:58.309391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe6f4d-af02-4753-82f8-a99648737c4d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.309673 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe6f4d-af02-4753-82f8-a99648737c4d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.309992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe6f4d-af02-4753-82f8-a99648737c4d None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.310653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe6f4d-af02-4753-82f8-a99648737c4d None admin] RBAC: Adding query filter params (name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:58.310960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe6f4d-af02-4753-82f8-a99648737c4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:58.312346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe6f4d-af02-4753-82f8-a99648737c4d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:58.318298 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c21ca-29e0-4506-9496-327e6256ee7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:58.319440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c21ca-29e0-4506-9496-327e6256ee7e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.319715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c21ca-29e0-4506-9496-327e6256ee7e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.319984 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c21ca-29e0-4506-9496-327e6256ee7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.320652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 289/568] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:32:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:58.321788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713a3af-a1ea-4db2-97ef-db7a295584cb None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.322079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713a3af-a1ea-4db2-97ef-db7a295584cb None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.322358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713a3af-a1ea-4db2-97ef-db7a295584cb None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.323669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 290/569] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:32:58.324298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713a3af-a1ea-4db2-97ef-db7a295584cb None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:58.331309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c21ca-29e0-4506-9496-327e6256ee7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:58.332749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c21ca-29e0-4506-9496-327e6256ee7e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-322c21ca-29e0-4506-9496-327e6256ee7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:32:58.344297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:58.345303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:58.346220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:32:58.346220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:32:58.346220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:32:58.346220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:32:58.346220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:32:58.346220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:32:58.346220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:32:58.346220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:58.346220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 280/570] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:58.346692 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c21ca-29e0-4506-9496-327e6256ee7e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:58.423783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:58.428236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713a3af-a1ea-4db2-97ef-db7a295584cb None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:58.445144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c21ca-29e0-4506-9496-327e6256ee7e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:58.461141 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd42bba-4042-467e-a19f-f7843063da98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:58.461762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd42bba-4042-467e-a19f-f7843063da98 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.462041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd42bba-4042-467e-a19f-f7843063da98 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.462309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd42bba-4042-467e-a19f-f7843063da98 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.462959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd42bba-4042-467e-a19f-f7843063da98 None admin] RBAC: Adding query filter params (name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:58.463261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd42bba-4042-467e-a19f-f7843063da98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:58.464618 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd42bba-4042-467e-a19f-f7843063da98 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:58.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 291/571] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:32:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:58.472908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd42bba-4042-467e-a19f-f7843063da98 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:58.478583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939ac6d-4ad7-47cd-bb9c-16172890218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:58.479441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939ac6d-4ad7-47cd-bb9c-16172890218e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.479713 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939ac6d-4ad7-47cd-bb9c-16172890218e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.479978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939ac6d-4ad7-47cd-bb9c-16172890218e None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.480606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939ac6d-4ad7-47cd-bb9c-16172890218e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:58.480905 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939ac6d-4ad7-47cd-bb9c-16172890218e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:58.482276 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939ac6d-4ad7-47cd-bb9c-16172890218e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:58.493878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 281/572] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:58.494863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160167d-0e21-495e-9f3a-7d5f5deec2af None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.495145 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160167d-0e21-495e-9f3a-7d5f5deec2af None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.495412 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160167d-0e21-495e-9f3a-7d5f5deec2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.512549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a160167d-0e21-495e-9f3a-7d5f5deec2af None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:32:58.513795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a160167d-0e21-495e-9f3a-7d5f5deec2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:32:58.527066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:58.573669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd42bba-4042-467e-a19f-f7843063da98 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:58.580883 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 282/573] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:32:58.581708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160167d-0e21-495e-9f3a-7d5f5deec2af None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:58.607363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e8161-b8b8-48d3-87c4-d902b0864c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:58.607971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e8161-b8b8-48d3-87c4-d902b0864c5d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.608555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e8161-b8b8-48d3-87c4-d902b0864c5d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.608555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e8161-b8b8-48d3-87c4-d902b0864c5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.619763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e8161-b8b8-48d3-87c4-d902b0864c5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:58.621060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e8161-b8b8-48d3-87c4-d902b0864c5d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-c97e8161-b8b8-48d3-87c4-d902b0864c5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:58.634798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:58.635748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:58.642184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:58.642184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:58.642184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:58.642184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 292/574] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:58.642184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e8161-b8b8-48d3-87c4-d902b0864c5d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:58.654881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:58.712079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160167d-0e21-495e-9f3a-7d5f5deec2af None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:58.743606 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e8161-b8b8-48d3-87c4-d902b0864c5d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:58.746879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5b9b4-8f5f-4557-b357-270be13ab259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:58.747489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5b9b4-8f5f-4557-b357-270be13ab259 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.747768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5b9b4-8f5f-4557-b357-270be13ab259 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.748038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5b9b4-8f5f-4557-b357-270be13ab259 None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.748735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5b9b4-8f5f-4557-b357-270be13ab259 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:58.749026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5b9b4-8f5f-4557-b357-270be13ab259 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:58.750727 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b5b9b4-8f5f-4557-b357-270be13ab259 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:58.766558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 283/575] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:32:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:58.766558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5b9b4-8f5f-4557-b357-270be13ab259 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:58.778635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:58.782749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90851718-7d11-4ca4-8121-85d2607a60a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:58.783378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90851718-7d11-4ca4-8121-85d2607a60a5 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.783658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90851718-7d11-4ca4-8121-85d2607a60a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.783982 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90851718-7d11-4ca4-8121-85d2607a60a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.795416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90851718-7d11-4ca4-8121-85d2607a60a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:58.796943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90851718-7d11-4ca4-8121-85d2607a60a5 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-90851718-7d11-4ca4-8121-85d2607a60a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:32:58.810656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:58.811622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:32:58.817816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:32:58.817816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:32:58.817816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:58.817816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 293/576] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:58.817816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90851718-7d11-4ca4-8121-85d2607a60a5 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:58.877636 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5b9b4-8f5f-4557-b357-270be13ab259 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:58.878835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:58.912569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a016dd-257e-47a7-ad00-c3c7fa9af840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:58.913185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a016dd-257e-47a7-ad00-c3c7fa9af840 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.913462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a016dd-257e-47a7-ad00-c3c7fa9af840 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.913750 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a016dd-257e-47a7-ad00-c3c7fa9af840 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.913961 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90851718-7d11-4ca4-8121-85d2607a60a5 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:58.920143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a016dd-257e-47a7-ad00-c3c7fa9af840 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:58.921560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a016dd-257e-47a7-ad00-c3c7fa9af840 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-77a016dd-257e-47a7-ad00-c3c7fa9af840 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:32:58.932948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:58.933850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:58.934719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:32:58.934719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:32:58.934719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:32:58.934719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:32:58.934719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:32:58.934719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:32:58.934719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:32:58.934719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:58.934719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 284/577] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:32:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:58.935209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a016dd-257e-47a7-ad00-c3c7fa9af840 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:58.951771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b4ea3-f12e-4872-9709-b915f9e7f01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:58.952374 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b4ea3-f12e-4872-9709-b915f9e7f01e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:58.952657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b4ea3-f12e-4872-9709-b915f9e7f01e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:58.952929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b4ea3-f12e-4872-9709-b915f9e7f01e None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:58.953565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b4ea3-f12e-4872-9709-b915f9e7f01e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:58.953894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b4ea3-f12e-4872-9709-b915f9e7f01e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:58.955268 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b4ea3-f12e-4872-9709-b915f9e7f01e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:58.966844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 294/578] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:58.967632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b4ea3-f12e-4872-9709-b915f9e7f01e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:59.005315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:59.039736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a016dd-257e-47a7-ad00-c3c7fa9af840 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:59.066513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b4ea3-f12e-4872-9709-b915f9e7f01e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:59.072380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521f3ec-7f51-47de-929e-4d90792eab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:59.073028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521f3ec-7f51-47de-929e-4d90792eab93 None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:59.073305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521f3ec-7f51-47de-929e-4d90792eab93 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:59.073581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a521f3ec-7f51-47de-929e-4d90792eab93 None admin] PATH_INFO: `/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/ee8a19f4d9704408a8b737095e5885f1/roles/a01bc9944e4f46a4a9a2b16f510541c9` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:59.087136 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521f3ec-7f51-47de-929e-4d90792eab93 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3df14a62714f6bab2acdfc979b655b, user_id=ee8a19f4d9704408a8b737095e5885f1, role_id=a01bc9944e4f46a4a9a2b16f510541c9)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:59.088565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a521f3ec-7f51-47de-929e-4d90792eab93 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:59.091176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:59.100758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93740750-e3c1-4afe-a40c-8ec86f2173af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:59.101357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93740750-e3c1-4afe-a40c-8ec86f2173af None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:59.101640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93740750-e3c1-4afe-a40c-8ec86f2173af None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:59.102002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93740750-e3c1-4afe-a40c-8ec86f2173af None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:59.102122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 285/579] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:32:58 2022] PUT /identity/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/ee8a19f4d9704408a8b737095e5885f1/roles/a01bc9944e4f46a4a9a2b16f510541c9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:32:59.102643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93740750-e3c1-4afe-a40c-8ec86f2173af None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:59.102808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521f3ec-7f51-47de-929e-4d90792eab93 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:59.103002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93740750-e3c1-4afe-a40c-8ec86f2173af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:59.104735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93740750-e3c1-4afe-a40c-8ec86f2173af None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:59.123589 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 295/580] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:59.124602 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93740750-e3c1-4afe-a40c-8ec86f2173af None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:59.231971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a521f3ec-7f51-47de-929e-4d90792eab93 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:59.246201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93740750-e3c1-4afe-a40c-8ec86f2173af None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:59.252714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:59.275928 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26626611-22e7-40cd-95d1-43e656ddb510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:59.275928 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26626611-22e7-40cd-95d1-43e656ddb510 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:59.276225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26626611-22e7-40cd-95d1-43e656ddb510 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:59.276460 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26626611-22e7-40cd-95d1-43e656ddb510 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:59.288416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26626611-22e7-40cd-95d1-43e656ddb510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:32:59.288956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a3f64-ee17-459b-b07d-268f69239b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:59.289566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a3f64-ee17-459b-b07d-268f69239b8b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:59.289875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a3f64-ee17-459b-b07d-268f69239b8b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:59.290146 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a3f64-ee17-459b-b07d-268f69239b8b None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:59.290838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a3f64-ee17-459b-b07d-268f69239b8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:59.291170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a3f64-ee17-459b-b07d-268f69239b8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:59.292603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a3f64-ee17-459b-b07d-268f69239b8b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-26626611-22e7-40cd-95d1-43e656ddb510 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:32:59.293182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:59.294096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:59.294922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 286/581] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:32:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:59.296388 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26626611-22e7-40cd-95d1-43e656ddb510 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:59.306376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 296/582] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:32:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:59.307214 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a3f64-ee17-459b-b07d-268f69239b8b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:59.358217 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:59.411610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26626611-22e7-40cd-95d1-43e656ddb510 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:59.418118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a3f64-ee17-459b-b07d-268f69239b8b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:59.449015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c9519-97bd-4312-ab7f-fb7af4146057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:59.449651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c9519-97bd-4312-ab7f-fb7af4146057 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:59.449940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c9519-97bd-4312-ab7f-fb7af4146057 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:59.450241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c9519-97bd-4312-ab7f-fb7af4146057 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:59.454446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0de2f9-0213-49f9-bdff-baac71095670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:59.455076 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0de2f9-0213-49f9-bdff-baac71095670 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:59.455355 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0de2f9-0213-49f9-bdff-baac71095670 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:59.455627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0de2f9-0213-49f9-bdff-baac71095670 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:59.457466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c9519-97bd-4312-ab7f-fb7af4146057 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:59.458947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c9519-97bd-4312-ab7f-fb7af4146057 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:59.463367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0de2f9-0213-49f9-bdff-baac71095670 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:59.464893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0de2f9-0213-49f9-bdff-baac71095670 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-b10c9519-97bd-4312-ab7f-fb7af4146057 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:32:59.471278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:59.472196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:59.473119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:59.473119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:59.473119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:32:59.473119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:32:59.473119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:32:59.473119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:32:59.473119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:32:59.473119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:32:59.473119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 08:32:59.473119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:32:59.473119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 287/583] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:59.474396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c9519-97bd-4312-ab7f-fb7af4146057 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-3a0de2f9-0213-49f9-bdff-baac71095670 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:32:59.482212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:32:59.483124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:32:59.485405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:32:59.485405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:32:59.485405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:32:59.485405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:32:59.485405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:32:59.485405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:32:59.485405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:32:59.485405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:32:59.485405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:32:59.485405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:32:59.485405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 297/584] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:32:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:32:59.485405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0de2f9-0213-49f9-bdff-baac71095670 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:59.520594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:59.586495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c9519-97bd-4312-ab7f-fb7af4146057 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:59.590394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0de2f9-0213-49f9-bdff-baac71095670 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:59.598070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:59.623828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004c6b5-1cc8-4271-a590-ca7ab48630f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:59.624558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004c6b5-1cc8-4271-a590-ca7ab48630f2 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:59.624834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004c6b5-1cc8-4271-a590-ca7ab48630f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:59.625103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004c6b5-1cc8-4271-a590-ca7ab48630f2 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:59.625791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004c6b5-1cc8-4271-a590-ca7ab48630f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:59.626149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004c6b5-1cc8-4271-a590-ca7ab48630f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:59.626770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e957182-1b4a-4ac8-9303-8d7f99a16bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:59.627366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e957182-1b4a-4ac8-9303-8d7f99a16bf2 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:59.627475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004c6b5-1cc8-4271-a590-ca7ab48630f2 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:59.627640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e957182-1b4a-4ac8-9303-8d7f99a16bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:59.627916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e957182-1b4a-4ac8-9303-8d7f99a16bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:59.628644 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e957182-1b4a-4ac8-9303-8d7f99a16bf2 None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=b232b71c77a24be5bfcdb6aeb5807f90) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:59.628952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e957182-1b4a-4ac8-9303-8d7f99a16bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:59.630483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e957182-1b4a-4ac8-9303-8d7f99a16bf2 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:59.635964 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 298/585] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:32:59 2022] GET /identity/v3/role_assignments?user.id=b232b71c77a24be5bfcdb6aeb5807f90&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:59.636897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 288/586] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:32:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:59.637999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004c6b5-1cc8-4271-a590-ca7ab48630f2 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:59.641223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e957182-1b4a-4ac8-9303-8d7f99a16bf2 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:59.776604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:59.781392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004c6b5-1cc8-4271-a590-ca7ab48630f2 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:59.788013 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e957182-1b4a-4ac8-9303-8d7f99a16bf2 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:59.822629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74f3d6-171f-44cb-bf9e-212dcad5ef8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:59.823246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74f3d6-171f-44cb-bf9e-212dcad5ef8f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:59.823525 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74f3d6-171f-44cb-bf9e-212dcad5ef8f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:59.823799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74f3d6-171f-44cb-bf9e-212dcad5ef8f None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:59.824535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74f3d6-171f-44cb-bf9e-212dcad5ef8f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:59.824842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74f3d6-171f-44cb-bf9e-212dcad5ef8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:59.826339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74f3d6-171f-44cb-bf9e-212dcad5ef8f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:59.839816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e273b0-44a0-4e86-99af-45683d71ed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:32:59.840348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e273b0-44a0-4e86-99af-45683d71ed6e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:32:59.840626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e273b0-44a0-4e86-99af-45683d71ed6e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:32:59.840899 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e273b0-44a0-4e86-99af-45683d71ed6e None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:32:59.841631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e273b0-44a0-4e86-99af-45683d71ed6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:32:59.841984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e273b0-44a0-4e86-99af-45683d71ed6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:32:59.843416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e273b0-44a0-4e86-99af-45683d71ed6e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:32:59.853323 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 289/587] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Oct 4 08:32:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:59.854519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74f3d6-171f-44cb-bf9e-212dcad5ef8f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:59.877334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 299/588] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Oct 4 08:32:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:32:59.877334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e273b0-44a0-4e86-99af-45683d71ed6e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:32:59.906280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:32:59.962884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74f3d6-171f-44cb-bf9e-212dcad5ef8f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:32:59.984186 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e273b0-44a0-4e86-99af-45683d71ed6e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:00.001663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f871c-4cc8-47c6-a7a6-77b48c5d358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:00.002368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f871c-4cc8-47c6-a7a6-77b48c5d358b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:00.002645 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f871c-4cc8-47c6-a7a6-77b48c5d358b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:00.002919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f871c-4cc8-47c6-a7a6-77b48c5d358b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:00.014160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f871c-4cc8-47c6-a7a6-77b48c5d358b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:00.015689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f871c-4cc8-47c6-a7a6-77b48c5d358b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:00.026484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f1fc4-4e85-47cc-9a39-26f39cf47372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:00.027399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f1fc4-4e85-47cc-9a39-26f39cf47372 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:00.027875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f1fc4-4e85-47cc-9a39-26f39cf47372 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:00.028361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f1fc4-4e85-47cc-9a39-26f39cf47372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-172f871c-4cc8-47c6-a7a6-77b48c5d358b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:00.030726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:00.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:00.032605 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:00.032605 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:00.032605 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:00.036458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 290/589] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:00.037845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f871c-4cc8-47c6-a7a6-77b48c5d358b None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:00.043082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f1fc4-4e85-47cc-9a39-26f39cf47372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:00.044874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f1fc4-4e85-47cc-9a39-26f39cf47372 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:00.050574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-469f1fc4-4e85-47cc-9a39-26f39cf47372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:00.063990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:00.064989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:00.065913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:00.065913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:00.065913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:00.068155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 300/590] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:00.069177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f1fc4-4e85-47cc-9a39-26f39cf47372 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:00.140947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f871c-4cc8-47c6-a7a6-77b48c5d358b None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:00.168717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:00.178898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bc1d4-ba40-453b-ab54-4053fa7a839b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:00.179648 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bc1d4-ba40-453b-ab54-4053fa7a839b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:00.179918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bc1d4-ba40-453b-ab54-4053fa7a839b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:00.180181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2bc1d4-ba40-453b-ab54-4053fa7a839b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:00.192758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bc1d4-ba40-453b-ab54-4053fa7a839b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:00.193362 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f1fc4-4e85-47cc-9a39-26f39cf47372 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:00.194356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2bc1d4-ba40-453b-ab54-4053fa7a839b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-cd2bc1d4-ba40-453b-ab54-4053fa7a839b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:00.209970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:00.210911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:00.211807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:00.211807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:00.211807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:00.211807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 291/591] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:00.212455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bc1d4-ba40-453b-ab54-4053fa7a839b None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:00.243836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade3227-80bc-4eeb-aa76-312f904b8d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:00.245160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade3227-80bc-4eeb-aa76-312f904b8d7d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:00.245643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade3227-80bc-4eeb-aa76-312f904b8d7d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:00.246407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade3227-80bc-4eeb-aa76-312f904b8d7d None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:00.247703 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade3227-80bc-4eeb-aa76-312f904b8d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:00.248447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade3227-80bc-4eeb-aa76-312f904b8d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:00.250954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade3227-80bc-4eeb-aa76-312f904b8d7d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:00.267816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 301/592] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:00.269171 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade3227-80bc-4eeb-aa76-312f904b8d7d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:00.291801 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:00.341315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2bc1d4-ba40-453b-ab54-4053fa7a839b None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:00.386048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20e989-597a-43c2-b4b4-8af7a84187d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:00.387117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20e989-597a-43c2-b4b4-8af7a84187d6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:00.387577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20e989-597a-43c2-b4b4-8af7a84187d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:00.388351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20e989-597a-43c2-b4b4-8af7a84187d6 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:00.389457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20e989-597a-43c2-b4b4-8af7a84187d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:00.390138 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20e989-597a-43c2-b4b4-8af7a84187d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:00.391761 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20e989-597a-43c2-b4b4-8af7a84187d6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:00.394276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade3227-80bc-4eeb-aa76-312f904b8d7d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:00.408044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 292/593] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:00.409244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20e989-597a-43c2-b4b4-8af7a84187d6 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:00.431827 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70555f-6262-47cb-ae56-f0e91e4a38dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:00.432544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70555f-6262-47cb-ae56-f0e91e4a38dc None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:00.432889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70555f-6262-47cb-ae56-f0e91e4a38dc None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:00.433230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70555f-6262-47cb-ae56-f0e91e4a38dc None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:00.434025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70555f-6262-47cb-ae56-f0e91e4a38dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:00.434395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70555f-6262-47cb-ae56-f0e91e4a38dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:00.436000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70555f-6262-47cb-ae56-f0e91e4a38dc None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:00.445647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:00.455312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 302/594] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:00.456213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70555f-6262-47cb-ae56-f0e91e4a38dc None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:00.522703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20e989-597a-43c2-b4b4-8af7a84187d6 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:00.549172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:00.562742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e366d-8a43-447f-b9a5-7bde23108a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:00.563630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e366d-8a43-447f-b9a5-7bde23108a31 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:00.564135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e366d-8a43-447f-b9a5-7bde23108a31 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:00.564604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e366d-8a43-447f-b9a5-7bde23108a31 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:00.565656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70555f-6262-47cb-ae56-f0e91e4a38dc None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:00.573929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e366d-8a43-447f-b9a5-7bde23108a31 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:00.575380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e366d-8a43-447f-b9a5-7bde23108a31 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-071e366d-8a43-447f-b9a5-7bde23108a31 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:00.594502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:00.595527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:00.596465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:00.596465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:00.596465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:00.596465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:00.596465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:00.596465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:00.596465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:00.596465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:00.596465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 08:33:00.596465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:00.599554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 293/595] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:33:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:00.601291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e366d-8a43-447f-b9a5-7bde23108a31 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:00.617324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee552b-e425-48ed-b19e-95c7167a9abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:00.619000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee552b-e425-48ed-b19e-95c7167a9abe None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:00.619493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee552b-e425-48ed-b19e-95c7167a9abe None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:00.619966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee552b-e425-48ed-b19e-95c7167a9abe None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:00.629467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee552b-e425-48ed-b19e-95c7167a9abe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:00.632356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee552b-e425-48ed-b19e-95c7167a9abe None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-9fee552b-e425-48ed-b19e-95c7167a9abe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:00.647237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:00.648089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:00.648963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:00.648963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:00.648963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:00.648963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:00.648963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:00.648963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:00.648963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:00.648963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:00.651535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 303/596] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:00.652503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee552b-e425-48ed-b19e-95c7167a9abe None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:00.724195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e366d-8a43-447f-b9a5-7bde23108a31 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:00.733130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:00.767661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7650ed4-5413-4af3-b28a-bf99a199a817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:00.768691 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650ed4-5413-4af3-b28a-bf99a199a817 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:00.769732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650ed4-5413-4af3-b28a-bf99a199a817 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:00.770207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650ed4-5413-4af3-b28a-bf99a199a817 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:00.778893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7650ed4-5413-4af3-b28a-bf99a199a817 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:00.780715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7650ed4-5413-4af3-b28a-bf99a199a817 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:00.782763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee552b-e425-48ed-b19e-95c7167a9abe None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-e7650ed4-5413-4af3-b28a-bf99a199a817 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:00.796207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:00.797196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:00.798108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:00.798108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:00.798108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:00.798108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:00.798108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:00.798108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:00.798108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:00.798108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:00.804214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 294/597] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:00.804214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7650ed4-5413-4af3-b28a-bf99a199a817 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:00.836377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88b3c0-819b-47aa-bc8b-85badb98a66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:00.837568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88b3c0-819b-47aa-bc8b-85badb98a66e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:00.838340 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88b3c0-819b-47aa-bc8b-85badb98a66e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:00.839204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88b3c0-819b-47aa-bc8b-85badb98a66e None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:00.840401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88b3c0-819b-47aa-bc8b-85badb98a66e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:00.840968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88b3c0-819b-47aa-bc8b-85badb98a66e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:00.843759 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88b3c0-819b-47aa-bc8b-85badb98a66e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:00.854591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:00.869746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 304/598] 10.222.0.26 () {56 vars in 1139 bytes} [Tue Oct 4 08:33:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:00.869746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88b3c0-819b-47aa-bc8b-85badb98a66e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:00.938640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7650ed4-5413-4af3-b28a-bf99a199a817 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:00.973581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88b3c0-819b-47aa-bc8b-85badb98a66e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:00.978559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b2efc-6892-4003-84ef-2075bdaae9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:00.979417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b2efc-6892-4003-84ef-2075bdaae9f1 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:00.979692 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b2efc-6892-4003-84ef-2075bdaae9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:00.979960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b2efc-6892-4003-84ef-2075bdaae9f1 None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:00.980641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b2efc-6892-4003-84ef-2075bdaae9f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:00.980943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b2efc-6892-4003-84ef-2075bdaae9f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:00.982421 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b2efc-6892-4003-84ef-2075bdaae9f1 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:00.996093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 295/599] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:00.997016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b2efc-6892-4003-84ef-2075bdaae9f1 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:00.999002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:01.018878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b0d1f-5228-4187-b577-ed4789876c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:01.019978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b0d1f-5228-4187-b577-ed4789876c1e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:01.020456 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b0d1f-5228-4187-b577-ed4789876c1e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:01.020960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b0d1f-5228-4187-b577-ed4789876c1e None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:01.022412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b0d1f-5228-4187-b577-ed4789876c1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:01.023009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b0d1f-5228-4187-b577-ed4789876c1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:01.024811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b0d1f-5228-4187-b577-ed4789876c1e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:01.038089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 305/600] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:01.039267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b0d1f-5228-4187-b577-ed4789876c1e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:01.103330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b2efc-6892-4003-84ef-2075bdaae9f1 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:01.105449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:01.140146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b339d9-5932-4730-8d19-0c43dbc12df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:01.140774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b339d9-5932-4730-8d19-0c43dbc12df1 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:01.141048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b339d9-5932-4730-8d19-0c43dbc12df1 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:01.141578 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b339d9-5932-4730-8d19-0c43dbc12df1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:01.151939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b339d9-5932-4730-8d19-0c43dbc12df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:01.153372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b339d9-5932-4730-8d19-0c43dbc12df1 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:01.154150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b0d1f-5228-4187-b577-ed4789876c1e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-e3b339d9-5932-4730-8d19-0c43dbc12df1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:01.168618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:01.169640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:01.170559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:01.170559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:01.170559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:01.174547 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 296/601] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:01.175542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b339d9-5932-4730-8d19-0c43dbc12df1 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:01.198442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e908eb4-a743-4035-933f-56b09583e7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:01.199507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e908eb4-a743-4035-933f-56b09583e7ab None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:01.200110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e908eb4-a743-4035-933f-56b09583e7ab None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:01.200609 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e908eb4-a743-4035-933f-56b09583e7ab None admin] PATH_INFO: `/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/6f224ed840244faea117fe97c29c26d1/roles/a01bc9944e4f46a4a9a2b16f510541c9` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:01.218474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e908eb4-a743-4035-933f-56b09583e7ab None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3df14a62714f6bab2acdfc979b655b, user_id=6f224ed840244faea117fe97c29c26d1, role_id=a01bc9944e4f46a4a9a2b16f510541c9)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:01.220372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e908eb4-a743-4035-933f-56b09583e7ab None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:01.237007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 306/602] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:33:01 2022] PUT /identity/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/6f224ed840244faea117fe97c29c26d1/roles/a01bc9944e4f46a4a9a2b16f510541c9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:33:01.238227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e908eb4-a743-4035-933f-56b09583e7ab None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:01.272854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:01.289071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b339d9-5932-4730-8d19-0c43dbc12df1 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:01.342251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccd732-0e10-49c4-aac2-ba4734f5a3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:01.342926 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccd732-0e10-49c4-aac2-ba4734f5a3d6 None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:01.343205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccd732-0e10-49c4-aac2-ba4734f5a3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:01.344073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccd732-0e10-49c4-aac2-ba4734f5a3d6 None admin] PATH_INFO: `/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/913a866ddc074b7f9aa142c70da98a69/roles/a01bc9944e4f46a4a9a2b16f510541c9` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:01.368395 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccd732-0e10-49c4-aac2-ba4734f5a3d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3df14a62714f6bab2acdfc979b655b, user_id=913a866ddc074b7f9aa142c70da98a69, role_id=a01bc9944e4f46a4a9a2b16f510541c9)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:01.370655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccd732-0e10-49c4-aac2-ba4734f5a3d6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:01.372524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e908eb4-a743-4035-933f-56b09583e7ab None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:01.393583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 297/603] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:33:01 2022] PUT /identity/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/913a866ddc074b7f9aa142c70da98a69/roles/a01bc9944e4f46a4a9a2b16f510541c9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:33:01.402821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef3122-3ee9-4fd6-b295-14697c7d64ac None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:01.403650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef3122-3ee9-4fd6-b295-14697c7d64ac None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:01.404128 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef3122-3ee9-4fd6-b295-14697c7d64ac None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:01.407585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 298/604] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:01.415146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfa221-c3a8-4028-b9e2-556d92a24cb3 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:01.415520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfa221-c3a8-4028-b9e2-556d92a24cb3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:01.415688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfa221-c3a8-4028-b9e2-556d92a24cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:01.426354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:01.444954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-eecfa221-c3a8-4028-b9e2-556d92a24cb3 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:01.447215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-eecfa221-c3a8-4028-b9e2-556d92a24cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:01.451037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800779ed-eebd-49fe-8c8b-2646bee9253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:01.451673 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800779ed-eebd-49fe-8c8b-2646bee9253a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:01.451863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800779ed-eebd-49fe-8c8b-2646bee9253a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:01.452138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800779ed-eebd-49fe-8c8b-2646bee9253a None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:01.452793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800779ed-eebd-49fe-8c8b-2646bee9253a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:01.453095 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800779ed-eebd-49fe-8c8b-2646bee9253a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:01.454523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800779ed-eebd-49fe-8c8b-2646bee9253a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:01.470979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 307/605] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:01.475471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800779ed-eebd-49fe-8c8b-2646bee9253a None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:01.572306 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 299/606] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:33:01.585013 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:01.606978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800779ed-eebd-49fe-8c8b-2646bee9253a None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:01.655253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a7935-97d5-4507-b82f-db337db14bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:01.655897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a7935-97d5-4507-b82f-db337db14bf2 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:01.656236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a7935-97d5-4507-b82f-db337db14bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:01.656514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a7935-97d5-4507-b82f-db337db14bf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:01.663881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a7935-97d5-4507-b82f-db337db14bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:01.665363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a7935-97d5-4507-b82f-db337db14bf2 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-0a1a7935-97d5-4507-b82f-db337db14bf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:01.679645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:01.680558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:01.681429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:01.681429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:01.681429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:01.681429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:01.681429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:01.681429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:01.681429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:01.681429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:01.681429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 308/607] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:01.686337 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfa221-c3a8-4028-b9e2-556d92a24cb3 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:01.807457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfa221-c3a8-4028-b9e2-556d92a24cb3 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:01.853438 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95822505-898a-4c7f-bb0b-1ac53c8797d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:01.854662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95822505-898a-4c7f-bb0b-1ac53c8797d2 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:01.855116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95822505-898a-4c7f-bb0b-1ac53c8797d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:01.855558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95822505-898a-4c7f-bb0b-1ac53c8797d2 None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:01.856434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95822505-898a-4c7f-bb0b-1ac53c8797d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:01.856912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95822505-898a-4c7f-bb0b-1ac53c8797d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:01.858596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95822505-898a-4c7f-bb0b-1ac53c8797d2 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:01.860332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:01.876708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 300/608] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:01.882573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a7935-97d5-4507-b82f-db337db14bf2 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:01.941886 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95822505-898a-4c7f-bb0b-1ac53c8797d2 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:02.023277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a7935-97d5-4507-b82f-db337db14bf2 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:02.071977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96931b91-bd3b-496f-ab41-8000642670d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:02.072569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96931b91-bd3b-496f-ab41-8000642670d9 None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:02.072853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96931b91-bd3b-496f-ab41-8000642670d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:02.073122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96931b91-bd3b-496f-ab41-8000642670d9 None admin] PATH_INFO: `/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/50d3b75cbc5c4d1686f4d93248e04746/roles/a01bc9944e4f46a4a9a2b16f510541c9` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:02.082804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95822505-898a-4c7f-bb0b-1ac53c8797d2 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:02.093064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96931b91-bd3b-496f-ab41-8000642670d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3df14a62714f6bab2acdfc979b655b, user_id=50d3b75cbc5c4d1686f4d93248e04746, role_id=a01bc9944e4f46a4a9a2b16f510541c9)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:02.095072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96931b91-bd3b-496f-ab41-8000642670d9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:02.105938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:02.115014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 309/609] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:33:01 2022] PUT /identity/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/50d3b75cbc5c4d1686f4d93248e04746/roles/a01bc9944e4f46a4a9a2b16f510541c9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:33:02.119035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91452b0d-86c9-4a7c-9eb2-84301c13c240 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:02.119624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91452b0d-86c9-4a7c-9eb2-84301c13c240 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:02.120117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91452b0d-86c9-4a7c-9eb2-84301c13c240 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:02.122206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 310/610] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:02.127996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5901a21-1ca1-413d-86d5-9c24cc40e01b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:02.129609 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5901a21-1ca1-413d-86d5-9c24cc40e01b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:02.131462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5901a21-1ca1-413d-86d5-9c24cc40e01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:02.152758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c18839-112e-446c-8ba8-83fdeac0f618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:02.157486 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c18839-112e-446c-8ba8-83fdeac0f618 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:02.158163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c18839-112e-446c-8ba8-83fdeac0f618 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:02.158631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c18839-112e-446c-8ba8-83fdeac0f618 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:02.170785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b5901a21-1ca1-413d-86d5-9c24cc40e01b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:02.172617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b5901a21-1ca1-413d-86d5-9c24cc40e01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:02.182340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c18839-112e-446c-8ba8-83fdeac0f618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-b4c18839-112e-446c-8ba8-83fdeac0f618 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:02.198690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:02.200286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:02.201147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:02.201147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:02.201147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:02.201147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:02.201147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:02.201147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:02.201147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:02.201147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:02.201147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:02.201147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:02.201147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:02.201147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:02.204648 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 301/611] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:33:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:02.210617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c18839-112e-446c-8ba8-83fdeac0f618 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:02.321780 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 311/612] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:02.323948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:02.374517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c18839-112e-446c-8ba8-83fdeac0f618 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:02.416729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50bc33-9dbe-41e4-a6c8-e9034c3afe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:02.417359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50bc33-9dbe-41e4-a6c8-e9034c3afe8e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:02.417622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50bc33-9dbe-41e4-a6c8-e9034c3afe8e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:02.417933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50bc33-9dbe-41e4-a6c8-e9034c3afe8e None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:02.418616 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50bc33-9dbe-41e4-a6c8-e9034c3afe8e None admin] RBAC: Adding query filter params (name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:02.418908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50bc33-9dbe-41e4-a6c8-e9034c3afe8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:02.420290 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50bc33-9dbe-41e4-a6c8-e9034c3afe8e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:02.429745 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 302/613] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:33:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:02.435382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5901a21-1ca1-413d-86d5-9c24cc40e01b None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:02.589506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5901a21-1ca1-413d-86d5-9c24cc40e01b None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:02.618059 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:02.633163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05486d90-b14d-4f35-a586-61d69b567dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:02.633827 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05486d90-b14d-4f35-a586-61d69b567dc1 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:02.634115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05486d90-b14d-4f35-a586-61d69b567dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:02.634392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05486d90-b14d-4f35-a586-61d69b567dc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:02.643586 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50bc33-9dbe-41e4-a6c8-e9034c3afe8e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:02.654247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05486d90-b14d-4f35-a586-61d69b567dc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:02.655875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05486d90-b14d-4f35-a586-61d69b567dc1 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-05486d90-b14d-4f35-a586-61d69b567dc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:02.673080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:02.674285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:02.675361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:02.675361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:02.675361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:02.678146 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 312/614] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:02.680249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05486d90-b14d-4f35-a586-61d69b567dc1 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:02.789440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50bc33-9dbe-41e4-a6c8-e9034c3afe8e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:02.789888 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05486d90-b14d-4f35-a586-61d69b567dc1 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:02.806712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:02.826078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c422e73-54e9-48a3-8a49-fad97d6f3382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:02.826932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c422e73-54e9-48a3-8a49-fad97d6f3382 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:02.827317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c422e73-54e9-48a3-8a49-fad97d6f3382 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:02.827694 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c422e73-54e9-48a3-8a49-fad97d6f3382 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:02.827843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c78fe-3c0b-4460-baf5-d4bdfe0f5e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:02.828455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c78fe-3c0b-4460-baf5-d4bdfe0f5e6d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:02.828737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c78fe-3c0b-4460-baf5-d4bdfe0f5e6d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:02.829010 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c78fe-3c0b-4460-baf5-d4bdfe0f5e6d None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:02.829730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c78fe-3c0b-4460-baf5-d4bdfe0f5e6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:02.830046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c78fe-3c0b-4460-baf5-d4bdfe0f5e6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:02.831427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c78fe-3c0b-4460-baf5-d4bdfe0f5e6d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:02.839613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c422e73-54e9-48a3-8a49-fad97d6f3382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:02.844132 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 313/615] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-8c422e73-54e9-48a3-8a49-fad97d6f3382 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:02.845147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:02.846309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:02.847348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:02.847348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:02.847348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:02.847348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:02.847348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:02.847348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:02.847348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:02.847348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:02.847348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:02.847348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:02.847348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:02.847348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:02.849283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 303/616] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:33:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:02.849483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c78fe-3c0b-4460-baf5-d4bdfe0f5e6d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:02.859150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c422e73-54e9-48a3-8a49-fad97d6f3382 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:02.901164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:02.954965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c78fe-3c0b-4460-baf5-d4bdfe0f5e6d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:02.991963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5262785-bf00-4426-a0d6-48168b16abf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:02.992607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5262785-bf00-4426-a0d6-48168b16abf5 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:02.992889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5262785-bf00-4426-a0d6-48168b16abf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:02.993163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5262785-bf00-4426-a0d6-48168b16abf5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:03.000463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5262785-bf00-4426-a0d6-48168b16abf5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:03.001984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5262785-bf00-4426-a0d6-48168b16abf5 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-f5262785-bf00-4426-a0d6-48168b16abf5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:03.013834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:03.014790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:03.016309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:03.016309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:03.016309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:03.016309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:03.016309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:03.016309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:03.016309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:03.016309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:03.016309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:33:03.016309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:03.016309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 314/617] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 08:33:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:03.020548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5262785-bf00-4426-a0d6-48168b16abf5 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:03.048957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c422e73-54e9-48a3-8a49-fad97d6f3382 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:03.115236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1a371-0d63-45dc-9fb2-25c78ba0a5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:03.116345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1a371-0d63-45dc-9fb2-25c78ba0a5e4 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:03.116628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1a371-0d63-45dc-9fb2-25c78ba0a5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:03.116902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1a371-0d63-45dc-9fb2-25c78ba0a5e4 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:03.117567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1a371-0d63-45dc-9fb2-25c78ba0a5e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:03.117898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1a371-0d63-45dc-9fb2-25c78ba0a5e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:03.119285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1a371-0d63-45dc-9fb2-25c78ba0a5e4 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:03.141179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 304/618] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:33:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:03.149311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1a371-0d63-45dc-9fb2-25c78ba0a5e4 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:03.197027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:03.202056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5262785-bf00-4426-a0d6-48168b16abf5 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:03.222084 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:03.277279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb645af1-7981-4f29-9fd0-8a8b50d4cf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:03.277932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb645af1-7981-4f29-9fd0-8a8b50d4cf37 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:03.278215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb645af1-7981-4f29-9fd0-8a8b50d4cf37 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:03.278487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb645af1-7981-4f29-9fd0-8a8b50d4cf37 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:03.279216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb645af1-7981-4f29-9fd0-8a8b50d4cf37 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:03.279524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb645af1-7981-4f29-9fd0-8a8b50d4cf37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:03.280970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb645af1-7981-4f29-9fd0-8a8b50d4cf37 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:03.317629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 315/619] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 08:33:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:03.326669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb645af1-7981-4f29-9fd0-8a8b50d4cf37 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:03.347715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1a371-0d63-45dc-9fb2-25c78ba0a5e4 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:03.453648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb645af1-7981-4f29-9fd0-8a8b50d4cf37 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:03.455321 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06db836-8961-4377-a0fd-1e0fc87bda08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:03.456187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06db836-8961-4377-a0fd-1e0fc87bda08 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:03.456663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06db836-8961-4377-a0fd-1e0fc87bda08 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:03.457130 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06db836-8961-4377-a0fd-1e0fc87bda08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:03.486739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06db836-8961-4377-a0fd-1e0fc87bda08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:03.488705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06db836-8961-4377-a0fd-1e0fc87bda08 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:03.497369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163f93b-9d6d-418b-8c37-64c1efb40174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:03.498015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163f93b-9d6d-418b-8c37-64c1efb40174 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:03.498302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163f93b-9d6d-418b-8c37-64c1efb40174 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:03.498579 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163f93b-9d6d-418b-8c37-64c1efb40174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-e06db836-8961-4377-a0fd-1e0fc87bda08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:03.504543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:03.505437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:03.506397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:03.506397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:03.506397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:03.508432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 305/620] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:03.513528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06db836-8961-4377-a0fd-1e0fc87bda08 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:03.560063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:03.646135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:03.659101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06db836-8961-4377-a0fd-1e0fc87bda08 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:03.662026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163f93b-9d6d-418b-8c37-64c1efb40174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:03.666223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163f93b-9d6d-418b-8c37-64c1efb40174 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:03.731707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225c745-396a-42d5-979c-5ef661d8ef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:03.732856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225c745-396a-42d5-979c-5ef661d8ef99 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:03.734619 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225c745-396a-42d5-979c-5ef661d8ef99 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:03.735125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225c745-396a-42d5-979c-5ef661d8ef99 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:03.737195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225c745-396a-42d5-979c-5ef661d8ef99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:03.738425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225c745-396a-42d5-979c-5ef661d8ef99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:03.739944 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225c745-396a-42d5-979c-5ef661d8ef99 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:03.758109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 306/621] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:03.768498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225c745-396a-42d5-979c-5ef661d8ef99 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:03.840880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 316/622] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:33:03.845906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163f93b-9d6d-418b-8c37-64c1efb40174 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:03.906655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:03.914175 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225c745-396a-42d5-979c-5ef661d8ef99 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:03.952985 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163f93b-9d6d-418b-8c37-64c1efb40174 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:03.957039 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136fb0d-ad3e-4dd7-9423-53dd0bf978e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:03.957818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136fb0d-ad3e-4dd7-9423-53dd0bf978e6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:03.958195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136fb0d-ad3e-4dd7-9423-53dd0bf978e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:03.958562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136fb0d-ad3e-4dd7-9423-53dd0bf978e6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:03.968057 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136fb0d-ad3e-4dd7-9423-53dd0bf978e6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:03.969866 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136fb0d-ad3e-4dd7-9423-53dd0bf978e6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-6136fb0d-ad3e-4dd7-9423-53dd0bf978e6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:03.984098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:03.985049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:03.986028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:03.986028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:03.986028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:03.986028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:03.986028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:03.986028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:03.986028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:03.986028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:03.986028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:33:03.986028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:03.989938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 307/623] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:33:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:03.992202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cbc20-e300-4550-aa79-e36d6947198b None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:03.992489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cbc20-e300-4550-aa79-e36d6947198b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:03.992767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cbc20-e300-4550-aa79-e36d6947198b None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:03.998743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 308/624] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:03.999717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cbc20-e300-4550-aa79-e36d6947198b None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:04.001678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f723a9-22cb-460a-9634-dbdc26e2aac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:04.002751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f723a9-22cb-460a-9634-dbdc26e2aac9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.003286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f723a9-22cb-460a-9634-dbdc26e2aac9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.003810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f723a9-22cb-460a-9634-dbdc26e2aac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.015185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:04.016780 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f723a9-22cb-460a-9634-dbdc26e2aac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:04.018671 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f723a9-22cb-460a-9634-dbdc26e2aac9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-50f723a9-22cb-460a-9634-dbdc26e2aac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:04.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:04.036377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:04.037277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:04.037277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:04.037277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:04.039642 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 317/625] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:04.041491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f087e70-2857-458c-bbbf-00bdcdfb3e36 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.041990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f087e70-2857-458c-bbbf-00bdcdfb3e36 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.042472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f087e70-2857-458c-bbbf-00bdcdfb3e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.063961 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-0f087e70-2857-458c-bbbf-00bdcdfb3e36 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:04.065798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-0f087e70-2857-458c-bbbf-00bdcdfb3e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:04.125871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cbc20-e300-4550-aa79-e36d6947198b None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:04.141462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 318/626] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:04.143062 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f087e70-2857-458c-bbbf-00bdcdfb3e36 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:04.184358 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7dbed5-7d6d-4a13-9e25-ceae27103ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:04.184977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dbed5-7d6d-4a13-9e25-ceae27103ba3 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.185259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dbed5-7d6d-4a13-9e25-ceae27103ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.185535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7dbed5-7d6d-4a13-9e25-ceae27103ba3 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.186327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7dbed5-7d6d-4a13-9e25-ceae27103ba3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:04.186637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7dbed5-7d6d-4a13-9e25-ceae27103ba3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:04.188078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7dbed5-7d6d-4a13-9e25-ceae27103ba3 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:04.204492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:04.207845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 309/627] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Oct 4 08:33:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:04.208927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df98b7f-3063-4067-a84c-2daa2876ff98 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.210425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df98b7f-3063-4067-a84c-2daa2876ff98 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.210714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df98b7f-3063-4067-a84c-2daa2876ff98 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.216348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 310/628] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:04.217221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df98b7f-3063-4067-a84c-2daa2876ff98 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:04.275921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f087e70-2857-458c-bbbf-00bdcdfb3e36 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:04.307493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:04.326340 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9431bd-9d4c-4862-bddd-9704a2b01990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:04.326950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9431bd-9d4c-4862-bddd-9704a2b01990 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.327226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9431bd-9d4c-4862-bddd-9704a2b01990 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.327494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9431bd-9d4c-4862-bddd-9704a2b01990 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.328135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9431bd-9d4c-4862-bddd-9704a2b01990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:04.328438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9431bd-9d4c-4862-bddd-9704a2b01990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:04.329826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9431bd-9d4c-4862-bddd-9704a2b01990 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:04.345721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 319/629] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:04.345721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfa2e5-9a11-4ea2-b370-802671a04c05 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.345721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfa2e5-9a11-4ea2-b370-802671a04c05 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.345721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfa2e5-9a11-4ea2-b370-802671a04c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.357749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df98b7f-3063-4067-a84c-2daa2876ff98 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:04.363425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-3dcfa2e5-9a11-4ea2-b370-802671a04c05 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:04.364638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3dcfa2e5-9a11-4ea2-b370-802671a04c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:04.406294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb70df-3d75-417d-b2e0-98e2b9d1b609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:04.407846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb70df-3d75-417d-b2e0-98e2b9d1b609 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.408343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb70df-3d75-417d-b2e0-98e2b9d1b609 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.409302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb70df-3d75-417d-b2e0-98e2b9d1b609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.424711 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb70df-3d75-417d-b2e0-98e2b9d1b609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:04.427471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb70df-3d75-417d-b2e0-98e2b9d1b609 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:04.441713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 320/630] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:04.441713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfa2e5-9a11-4ea2-b370-802671a04c05 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-d1cb70df-3d75-417d-b2e0-98e2b9d1b609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:04.455601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:04.456565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:04.457462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:04.457462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:04.457462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:04.457462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 311/631] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:04.458135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb70df-3d75-417d-b2e0-98e2b9d1b609 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:04.477971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:04.561809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfa2e5-9a11-4ea2-b370-802671a04c05 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:04.574349 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb70df-3d75-417d-b2e0-98e2b9d1b609 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:04.594053 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:04.605935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df44ce-80dc-496d-8454-d062562a8595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:04.606878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df44ce-80dc-496d-8454-d062562a8595 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.607342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df44ce-80dc-496d-8454-d062562a8595 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.607754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df44ce-80dc-496d-8454-d062562a8595 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.611517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5c468-d390-498c-bc04-25e9fbcc9075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:04.612312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5c468-d390-498c-bc04-25e9fbcc9075 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.612884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5c468-d390-498c-bc04-25e9fbcc9075 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.613324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5c468-d390-498c-bc04-25e9fbcc9075 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.614841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df44ce-80dc-496d-8454-d062562a8595 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:04.616827 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df44ce-80dc-496d-8454-d062562a8595 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:04.626258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5c468-d390-498c-bc04-25e9fbcc9075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-36df44ce-80dc-496d-8454-d062562a8595 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:04.631916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:04.632956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:04.634072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:04.634072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:04.634072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:04.634072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:04.634072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:04.634072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:04.634072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:04.634072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:04.636074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 321/632] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:04.636972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df44ce-80dc-496d-8454-d062562a8595 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-77f5c468-d390-498c-bc04-25e9fbcc9075 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:04.638466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:04.639541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:04.640553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:04.640553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:04.640553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:04.640553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:04.640553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:04.640553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:04.640553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:04.640553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:04.640553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:04.640553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:04.640553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:04.640553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:04.645751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 312/633] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:33:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:04.647049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5c468-d390-498c-bc04-25e9fbcc9075 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:04.756761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:04.767989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df44ce-80dc-496d-8454-d062562a8595 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:04.774396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5c468-d390-498c-bc04-25e9fbcc9075 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:04.801225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b290f-9912-43d8-bac7-5905ad63cf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:04.801854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b290f-9912-43d8-bac7-5905ad63cf7c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.802137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b290f-9912-43d8-bac7-5905ad63cf7c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.802411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b290f-9912-43d8-bac7-5905ad63cf7c None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.803584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b290f-9912-43d8-bac7-5905ad63cf7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:04.803886 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b290f-9912-43d8-bac7-5905ad63cf7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:04.805249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b290f-9912-43d8-bac7-5905ad63cf7c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:04.808659 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba6dc-bc4b-40bb-9bac-0c9901363d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:04.809253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba6dc-bc4b-40bb-9bac-0c9901363d42 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.809527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba6dc-bc4b-40bb-9bac-0c9901363d42 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.809844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba6dc-bc4b-40bb-9bac-0c9901363d42 None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.810519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bba6dc-bc4b-40bb-9bac-0c9901363d42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:04.810820 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bba6dc-bc4b-40bb-9bac-0c9901363d42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:04.812278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bba6dc-bc4b-40bb-9bac-0c9901363d42 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:04.816825 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 322/634] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:04.817453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b290f-9912-43d8-bac7-5905ad63cf7c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:04.826719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 313/635] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:04.827513 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba6dc-bc4b-40bb-9bac-0c9901363d42 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:04.867130 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:04.918707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b290f-9912-43d8-bac7-5905ad63cf7c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:04.924516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba6dc-bc4b-40bb-9bac-0c9901363d42 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:04.951434 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a55cdc-0357-44fd-9f87-621cd35cd5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:04.952031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a55cdc-0357-44fd-9f87-621cd35cd5cb None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.952308 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a55cdc-0357-44fd-9f87-621cd35cd5cb None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.952577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a55cdc-0357-44fd-9f87-621cd35cd5cb None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.953225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a55cdc-0357-44fd-9f87-621cd35cd5cb None admin] RBAC: Adding query filter params (name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:04.953525 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a55cdc-0357-44fd-9f87-621cd35cd5cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:04.954916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a55cdc-0357-44fd-9f87-621cd35cd5cb None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:04.958176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318d9d6-4eba-46be-a6a6-a004b296c38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:04.958785 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318d9d6-4eba-46be-a6a6-a004b296c38a None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.959611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318d9d6-4eba-46be-a6a6-a004b296c38a None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.959877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318d9d6-4eba-46be-a6a6-a004b296c38a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.963501 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 323/636] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:33:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:04.964742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8098a-f185-4389-997e-70374f9e4a39 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:04.965029 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8098a-f185-4389-997e-70374f9e4a39 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:04.965487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8098a-f185-4389-997e-70374f9e4a39 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:04.966865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 324/637] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:04.969897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318d9d6-4eba-46be-a6a6-a004b296c38a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:04.970444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8098a-f185-4389-997e-70374f9e4a39 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-c318d9d6-4eba-46be-a6a6-a004b296c38a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:04.974541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:04.976225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 314/638] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:33:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:04.977071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318d9d6-4eba-46be-a6a6-a004b296c38a None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:04.986231 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:05.072898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8098a-f185-4389-997e-70374f9e4a39 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:05.073412 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:05.076419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318d9d6-4eba-46be-a6a6-a004b296c38a None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:05.107427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f207c5-c08b-4b0f-a6ac-e4b6b724cff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:05.108047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f207c5-c08b-4b0f-a6ac-e4b6b724cff8 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:05.108327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f207c5-c08b-4b0f-a6ac-e4b6b724cff8 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:05.108600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f207c5-c08b-4b0f-a6ac-e4b6b724cff8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:05.110033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777e5e3-54f6-4a96-87db-add6a9af6a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:05.110623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777e5e3-54f6-4a96-87db-add6a9af6a20 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:05.110897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777e5e3-54f6-4a96-87db-add6a9af6a20 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:05.111170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777e5e3-54f6-4a96-87db-add6a9af6a20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:05.111907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5777e5e3-54f6-4a96-87db-add6a9af6a20 None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=ee8a19f4d9704408a8b737095e5885f1) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:05.112212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5777e5e3-54f6-4a96-87db-add6a9af6a20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:05.113661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5777e5e3-54f6-4a96-87db-add6a9af6a20 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:05.115025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f207c5-c08b-4b0f-a6ac-e4b6b724cff8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:05.116474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f207c5-c08b-4b0f-a6ac-e4b6b724cff8 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:05.119304 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 315/639] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:04 2022] GET /identity/v3/role_assignments?user.id=ee8a19f4d9704408a8b737095e5885f1&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:05.120089 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777e5e3-54f6-4a96-87db-add6a9af6a20 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-28f207c5-c08b-4b0f-a6ac-e4b6b724cff8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:05.127488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:05.128430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:05.129306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:05.129306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:05.129306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:05.129306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:05.129306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:05.129306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:05.129306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:05.129306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:05.129306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 325/640] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:05.132169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80372095-0d57-4c8c-9157-aa596e39152a None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:05.132169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80372095-0d57-4c8c-9157-aa596e39152a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:05.132169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80372095-0d57-4c8c-9157-aa596e39152a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:05.151881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-80372095-0d57-4c8c-9157-aa596e39152a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:05.153132 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-80372095-0d57-4c8c-9157-aa596e39152a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:05.233957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 326/641] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:05.233957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80372095-0d57-4c8c-9157-aa596e39152a None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:05.238553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:05.243340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777e5e3-54f6-4a96-87db-add6a9af6a20 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:05.286961 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f3018-32b9-4082-8f91-b1ea28ef1ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:05.287931 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f3018-32b9-4082-8f91-b1ea28ef1ef5 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:05.288383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f3018-32b9-4082-8f91-b1ea28ef1ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:05.288651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f3018-32b9-4082-8f91-b1ea28ef1ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:05.304073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f3018-32b9-4082-8f91-b1ea28ef1ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:05.306438 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f3018-32b9-4082-8f91-b1ea28ef1ef5 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-ea8f3018-32b9-4082-8f91-b1ea28ef1ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:05.325063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:05.326535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:05.327580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:05.327580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:05.327580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:05.329884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 316/642] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:05.330948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f3018-32b9-4082-8f91-b1ea28ef1ef5 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:05.351302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:05.370627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80372095-0d57-4c8c-9157-aa596e39152a None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:05.416378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36183e96-2326-434d-9364-e7a763f9949c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:05.416999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36183e96-2326-434d-9364-e7a763f9949c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:05.417349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36183e96-2326-434d-9364-e7a763f9949c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:05.417628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36183e96-2326-434d-9364-e7a763f9949c None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:05.418387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36183e96-2326-434d-9364-e7a763f9949c None admin] RBAC: Adding query filter params (name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:05.418643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36183e96-2326-434d-9364-e7a763f9949c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:05.420016 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36183e96-2326-434d-9364-e7a763f9949c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:05.429823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 327/643] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:33:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:05.430942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36183e96-2326-434d-9364-e7a763f9949c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:05.444517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f3018-32b9-4082-8f91-b1ea28ef1ef5 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:05.481534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d405f-fad1-4162-bb76-9e1f8196f59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:05.482329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d405f-fad1-4162-bb76-9e1f8196f59e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:05.482628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d405f-fad1-4162-bb76-9e1f8196f59e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:05.482903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d405f-fad1-4162-bb76-9e1f8196f59e None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:05.483982 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d405f-fad1-4162-bb76-9e1f8196f59e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:05.484572 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d405f-fad1-4162-bb76-9e1f8196f59e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:05.486212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d405f-fad1-4162-bb76-9e1f8196f59e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:05.501322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:05.502489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 317/644] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:05.503371 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d405f-fad1-4162-bb76-9e1f8196f59e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:05.546587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36183e96-2326-434d-9364-e7a763f9949c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:05.582942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db4c55-e8ad-4594-b002-b5702226b6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:05.583440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:05.583845 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db4c55-e8ad-4594-b002-b5702226b6c9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:05.584390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db4c55-e8ad-4594-b002-b5702226b6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:05.584500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db4c55-e8ad-4594-b002-b5702226b6c9 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:05.585093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db4c55-e8ad-4594-b002-b5702226b6c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:05.585420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db4c55-e8ad-4594-b002-b5702226b6c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:05.587179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db4c55-e8ad-4594-b002-b5702226b6c9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:05.603382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 328/645] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:05.603382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db4c55-e8ad-4594-b002-b5702226b6c9 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:05.613750 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d405f-fad1-4162-bb76-9e1f8196f59e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:05.652851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a96366-a589-4696-9ccc-099c648ffe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:05.653607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a96366-a589-4696-9ccc-099c648ffe8f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:05.653841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a96366-a589-4696-9ccc-099c648ffe8f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:05.654118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a96366-a589-4696-9ccc-099c648ffe8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:05.664999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a96366-a589-4696-9ccc-099c648ffe8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:05.666465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a96366-a589-4696-9ccc-099c648ffe8f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-a0a96366-a589-4696-9ccc-099c648ffe8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:05.682200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:05.683144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:05.684229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:05.684229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:05.684229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:05.687909 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 318/646] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:05.688546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a96366-a589-4696-9ccc-099c648ffe8f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:05.710890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db4c55-e8ad-4594-b002-b5702226b6c9 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:05.741619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:05.755221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44bd07-40ef-4429-9c51-b205a084b27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:05.756495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44bd07-40ef-4429-9c51-b205a084b27d None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:05.756778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44bd07-40ef-4429-9c51-b205a084b27d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:05.757039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44bd07-40ef-4429-9c51-b205a084b27d None admin] PATH_INFO: `/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/b232b71c77a24be5bfcdb6aeb5807f90/roles/a01bc9944e4f46a4a9a2b16f510541c9` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:05.772029 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44bd07-40ef-4429-9c51-b205a084b27d None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3df14a62714f6bab2acdfc979b655b, user_id=b232b71c77a24be5bfcdb6aeb5807f90, role_id=a01bc9944e4f46a4a9a2b16f510541c9)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:05.773481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44bd07-40ef-4429-9c51-b205a084b27d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:05.785720 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 329/647] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:33:05 2022] PUT /identity/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/b232b71c77a24be5bfcdb6aeb5807f90/roles/a01bc9944e4f46a4a9a2b16f510541c9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:33:05.786506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44bd07-40ef-4429-9c51-b205a084b27d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:05.817517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a96366-a589-4696-9ccc-099c648ffe8f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:05.856225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:05.865392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f929e-5cf4-4d94-88c8-2bf8b1174be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:05.866172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f929e-5cf4-4d94-88c8-2bf8b1174be0 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:05.866599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f929e-5cf4-4d94-88c8-2bf8b1174be0 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:05.866948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f929e-5cf4-4d94-88c8-2bf8b1174be0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:05.884850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f929e-5cf4-4d94-88c8-2bf8b1174be0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-c67f929e-5cf4-4d94-88c8-2bf8b1174be0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:05.892303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:05.893374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:05.894315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:05.894315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:05.894315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:05.894315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:05.894315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:05.894315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:05.894315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:05.894315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:05.894315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:05.894315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:05.894315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:05.894315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:05.896177 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 319/648] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:33:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:05.898005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f929e-5cf4-4d94-88c8-2bf8b1174be0 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:05.913420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44bd07-40ef-4429-9c51-b205a084b27d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:05.952372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d69ce-8033-47b2-82e5-d1861f6b0112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:05.953053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d69ce-8033-47b2-82e5-d1861f6b0112 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:05.953333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d69ce-8033-47b2-82e5-d1861f6b0112 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:05.953608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d69ce-8033-47b2-82e5-d1861f6b0112 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:05.963076 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d69ce-8033-47b2-82e5-d1861f6b0112 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:05.964618 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d69ce-8033-47b2-82e5-d1861f6b0112 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-177d69ce-8033-47b2-82e5-d1861f6b0112 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:05.978657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:05.979932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:05.980874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:05.980874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:05.980874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:05.980874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:05.980874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:05.980874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:05.980874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:05.980874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:05.980874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:33:05.980874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:05.980874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 330/649] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:33:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:05.981511 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d69ce-8033-47b2-82e5-d1861f6b0112 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:06.031147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:06.038609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f929e-5cf4-4d94-88c8-2bf8b1174be0 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:06.083197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cb825-c0c1-465d-9f72-b57f35e65a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:06.083804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cb825-c0c1-465d-9f72-b57f35e65a91 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:06.084138 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cb825-c0c1-465d-9f72-b57f35e65a91 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:06.084412 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cb825-c0c1-465d-9f72-b57f35e65a91 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:06.085044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cb825-c0c1-465d-9f72-b57f35e65a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:06.085343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cb825-c0c1-465d-9f72-b57f35e65a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:06.087249 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cb825-c0c1-465d-9f72-b57f35e65a91 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:06.104903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 320/650] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:06.105658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cb825-c0c1-465d-9f72-b57f35e65a91 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:06.129214 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d69ce-8033-47b2-82e5-d1861f6b0112 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:06.158860 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:06.181783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dee00-6626-4898-9b9f-171f9ffee17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:06.182675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dee00-6626-4898-9b9f-171f9ffee17f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:06.183376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dee00-6626-4898-9b9f-171f9ffee17f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:06.183807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305dee00-6626-4898-9b9f-171f9ffee17f None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:06.185507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dee00-6626-4898-9b9f-171f9ffee17f None admin] RBAC: Adding query filter params (name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:06.185994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dee00-6626-4898-9b9f-171f9ffee17f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:06.187525 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305dee00-6626-4898-9b9f-171f9ffee17f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:06.199511 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 331/651] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:33:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:06.200556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dee00-6626-4898-9b9f-171f9ffee17f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:06.221383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cb825-c0c1-465d-9f72-b57f35e65a91 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:06.259181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e007a-0f47-4f58-8663-40d26c4612be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:06.260038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e007a-0f47-4f58-8663-40d26c4612be None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:06.260458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e007a-0f47-4f58-8663-40d26c4612be None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:06.260823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e007a-0f47-4f58-8663-40d26c4612be None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:06.261640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e007a-0f47-4f58-8663-40d26c4612be None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:06.262067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e007a-0f47-4f58-8663-40d26c4612be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:06.264045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e007a-0f47-4f58-8663-40d26c4612be None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:06.282864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 321/652] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Oct 4 08:33:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:06.283896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e007a-0f47-4f58-8663-40d26c4612be None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:06.324382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:06.331717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305dee00-6626-4898-9b9f-171f9ffee17f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:06.369052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc45920-bce2-4611-ab11-e6bf30d5fa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:06.370067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc45920-bce2-4611-ab11-e6bf30d5fa9a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:06.370510 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc45920-bce2-4611-ab11-e6bf30d5fa9a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:06.371048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc45920-bce2-4611-ab11-e6bf30d5fa9a None admin] PATH_INFO: `/v3/users/glance` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:06.379329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc45920-bce2-4611-ab11-e6bf30d5fa9a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:06.381295 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc45920-bce2-4611-ab11-e6bf30d5fa9a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:06.389516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e007a-0f47-4f58-8663-40d26c4612be None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-2fc45920-bce2-4611-ab11-e6bf30d5fa9a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:06.394122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:06.395082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:06.396120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:06.396120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:06.396120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:06.396120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:06.396120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:06.396120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:06.396120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:06.396120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:06.396120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 08:33:06.396120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:06.398044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:06.399668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 332/653] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:33:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:06.400980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc45920-bce2-4611-ab11-e6bf30d5fa9a None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:06.433107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b38bf-729c-437f-b3da-e2b97de9c1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:06.433751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b38bf-729c-437f-b3da-e2b97de9c1cb None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:06.434032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b38bf-729c-437f-b3da-e2b97de9c1cb None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:06.434299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b38bf-729c-437f-b3da-e2b97de9c1cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:06.445285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b38bf-729c-437f-b3da-e2b97de9c1cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:06.446903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b38bf-729c-437f-b3da-e2b97de9c1cb None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-bb2b38bf-729c-437f-b3da-e2b97de9c1cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:06.463216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:06.464607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:06.465495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:06.465495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:06.465495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:06.469945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 322/654] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:06.470808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b38bf-729c-437f-b3da-e2b97de9c1cb None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:06.517365 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc45920-bce2-4611-ab11-e6bf30d5fa9a None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:06.555756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bf38d-4081-4486-a7ee-439dd042286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:06.556898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bf38d-4081-4486-a7ee-439dd042286f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:06.557535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bf38d-4081-4486-a7ee-439dd042286f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:06.558052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bf38d-4081-4486-a7ee-439dd042286f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:06.580799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b38bf-729c-437f-b3da-e2b97de9c1cb None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:06.587035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:06.619932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f9906-95da-4cc5-9e1d-28a5b6fe63d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:06.620919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f9906-95da-4cc5-9e1d-28a5b6fe63d9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:06.621457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f9906-95da-4cc5-9e1d-28a5b6fe63d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:06.621945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f9906-95da-4cc5-9e1d-28a5b6fe63d9 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:06.622858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f9906-95da-4cc5-9e1d-28a5b6fe63d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:06.623333 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f9906-95da-4cc5-9e1d-28a5b6fe63d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:06.625211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f9906-95da-4cc5-9e1d-28a5b6fe63d9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:06.635045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bf38d-4081-4486-a7ee-439dd042286f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:06.636597 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6bf38d-4081-4486-a7ee-439dd042286f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:06.649646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 323/655] 10.222.0.26 () {56 vars in 1137 bytes} [Tue Oct 4 08:33:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:06.650940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f9906-95da-4cc5-9e1d-28a5b6fe63d9 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:06.674579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:06.753314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 333/656] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:33:06.754277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bf38d-4081-4486-a7ee-439dd042286f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:06.766131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f9906-95da-4cc5-9e1d-28a5b6fe63d9 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:06.804045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf2ada-d2ff-46ce-b650-8a05b4b3f675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:06.805054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf2ada-d2ff-46ce-b650-8a05b4b3f675 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:06.805545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf2ada-d2ff-46ce-b650-8a05b4b3f675 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:06.806044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf2ada-d2ff-46ce-b650-8a05b4b3f675 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:06.806977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf2ada-d2ff-46ce-b650-8a05b4b3f675 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:06.807577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf2ada-d2ff-46ce-b650-8a05b4b3f675 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:06.809134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf2ada-d2ff-46ce-b650-8a05b4b3f675 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:06.814083 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:06.829282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 324/657] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:06.830566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf2ada-d2ff-46ce-b650-8a05b4b3f675 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:06.866919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bf38d-4081-4486-a7ee-439dd042286f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:06.904106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c78446-d885-44b6-a17e-b66036995cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:06.904664 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c78446-d885-44b6-a17e-b66036995cba None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:06.904947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c78446-d885-44b6-a17e-b66036995cba None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:06.905222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c78446-d885-44b6-a17e-b66036995cba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:06.914937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:06.941471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf2ada-d2ff-46ce-b650-8a05b4b3f675 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:06.978376 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033513e-4e18-44d5-a2b3-96e90e864ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:06.979425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033513e-4e18-44d5-a2b3-96e90e864ee2 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:06.979796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033513e-4e18-44d5-a2b3-96e90e864ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:06.980246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033513e-4e18-44d5-a2b3-96e90e864ee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:06.981472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c78446-d885-44b6-a17e-b66036995cba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:06.983350 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c78446-d885-44b6-a17e-b66036995cba None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:06.992100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033513e-4e18-44d5-a2b3-96e90e864ee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:06.993825 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033513e-4e18-44d5-a2b3-96e90e864ee2 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-4033513e-4e18-44d5-a2b3-96e90e864ee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:07.009993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:07.011190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:07.012166 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:07.012166 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:07.012166 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:07.012968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 325/658] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:07.013861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033513e-4e18-44d5-a2b3-96e90e864ee2 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:07.046811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:07.100304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 334/659] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:33:07.100304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c78446-d885-44b6-a17e-b66036995cba None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:07.129282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033513e-4e18-44d5-a2b3-96e90e864ee2 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:07.158075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:07.167714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfea3d-d7f7-4eb7-b798-0b766009ce12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:07.168593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfea3d-d7f7-4eb7-b798-0b766009ce12 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:07.169043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfea3d-d7f7-4eb7-b798-0b766009ce12 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:07.169495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfea3d-d7f7-4eb7-b798-0b766009ce12 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:07.178593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfea3d-d7f7-4eb7-b798-0b766009ce12 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:07.180274 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfea3d-d7f7-4eb7-b798-0b766009ce12 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-eabfea3d-d7f7-4eb7-b798-0b766009ce12 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:07.192560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:07.193597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:07.194561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:07.194561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:07.194561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:07.194561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:07.194561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:07.194561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:07.194561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:07.194561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:07.194561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 08:33:07.194561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:07.201292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 326/660] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:33:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:07.202507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfea3d-d7f7-4eb7-b798-0b766009ce12 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:07.208216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c78446-d885-44b6-a17e-b66036995cba None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:07.246310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9458b0-0f70-4d09-9ac4-6d596918893b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:07.247046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9458b0-0f70-4d09-9ac4-6d596918893b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:07.247454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9458b0-0f70-4d09-9ac4-6d596918893b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:07.247878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9458b0-0f70-4d09-9ac4-6d596918893b None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:07.248743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9458b0-0f70-4d09-9ac4-6d596918893b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:07.249153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9458b0-0f70-4d09-9ac4-6d596918893b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:07.250815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9458b0-0f70-4d09-9ac4-6d596918893b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:07.263655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 335/661] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:07.264864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9458b0-0f70-4d09-9ac4-6d596918893b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:07.284563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:07.312945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfea3d-d7f7-4eb7-b798-0b766009ce12 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:07.351640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec339d-3dd1-48c3-8fbe-7bea24f72961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:07.352768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec339d-3dd1-48c3-8fbe-7bea24f72961 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:07.353349 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec339d-3dd1-48c3-8fbe-7bea24f72961 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:07.353876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec339d-3dd1-48c3-8fbe-7bea24f72961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:07.366779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec339d-3dd1-48c3-8fbe-7bea24f72961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:07.368659 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec339d-3dd1-48c3-8fbe-7bea24f72961 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:07.379034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9458b0-0f70-4d09-9ac4-6d596918893b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-29ec339d-3dd1-48c3-8fbe-7bea24f72961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:07.385019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:07.385955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:07.386813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:07.386813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:07.386813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:07.388585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 327/662] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:07.389568 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec339d-3dd1-48c3-8fbe-7bea24f72961 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:07.417053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219fb5e-7088-4803-b75d-90c592acdf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:07.417584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219fb5e-7088-4803-b75d-90c592acdf39 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:07.417883 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219fb5e-7088-4803-b75d-90c592acdf39 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:07.418225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219fb5e-7088-4803-b75d-90c592acdf39 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:07.419034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219fb5e-7088-4803-b75d-90c592acdf39 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:07.419342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219fb5e-7088-4803-b75d-90c592acdf39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:07.421045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219fb5e-7088-4803-b75d-90c592acdf39 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:07.434847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:07.442012 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 336/663] 10.222.0.26 () {56 vars in 1139 bytes} [Tue Oct 4 08:33:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:07.443083 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219fb5e-7088-4803-b75d-90c592acdf39 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:07.505103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec339d-3dd1-48c3-8fbe-7bea24f72961 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:07.522694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:07.544186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f66d6-ac57-44c7-ba89-302356fadd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:07.544873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f66d6-ac57-44c7-ba89-302356fadd11 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:07.545155 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f66d6-ac57-44c7-ba89-302356fadd11 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:07.545420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f66d6-ac57-44c7-ba89-302356fadd11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:07.551287 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219fb5e-7088-4803-b75d-90c592acdf39 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:07.553634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f66d6-ac57-44c7-ba89-302356fadd11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:07.555148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5f66d6-ac57-44c7-ba89-302356fadd11 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-2d5f66d6-ac57-44c7-ba89-302356fadd11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:07.567959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:07.569106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:07.570037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:07.570037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:07.570037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:07.570037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:07.570037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:07.570037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:07.570037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:07.570037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:07.570037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 328/664] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:07.570037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f66d6-ac57-44c7-ba89-302356fadd11 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:07.592937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad79452-cccd-4d37-aa8e-bf72c42fedd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:07.593922 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad79452-cccd-4d37-aa8e-bf72c42fedd9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:07.594273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad79452-cccd-4d37-aa8e-bf72c42fedd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:07.594614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad79452-cccd-4d37-aa8e-bf72c42fedd9 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:07.595337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad79452-cccd-4d37-aa8e-bf72c42fedd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:07.595721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad79452-cccd-4d37-aa8e-bf72c42fedd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:07.597183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad79452-cccd-4d37-aa8e-bf72c42fedd9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:07.615992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 337/665] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:07.615992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f34b4-6d96-4929-9d7a-15f55bda5106 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:07.615992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f34b4-6d96-4929-9d7a-15f55bda5106 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:07.615992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f34b4-6d96-4929-9d7a-15f55bda5106 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:07.615992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 338/666] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:07.615992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f34b4-6d96-4929-9d7a-15f55bda5106 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:07.682229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:07.686088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5f66d6-ac57-44c7-ba89-302356fadd11 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:07.723260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f34b4-6d96-4929-9d7a-15f55bda5106 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:07.723551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a7253-887b-464c-9059-e74b3e64e15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:07.724116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a7253-887b-464c-9059-e74b3e64e15c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:07.724397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a7253-887b-464c-9059-e74b3e64e15c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:07.724835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a7253-887b-464c-9059-e74b3e64e15c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:07.759973 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e448b-513b-4883-be55-2364c964293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:07.760575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e448b-513b-4883-be55-2364c964293a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:07.760851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e448b-513b-4883-be55-2364c964293a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:07.761123 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e448b-513b-4883-be55-2364c964293a None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:07.761902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e448b-513b-4883-be55-2364c964293a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:07.762207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e448b-513b-4883-be55-2364c964293a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:07.763642 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e448b-513b-4883-be55-2364c964293a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:07.765330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:07.782477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 339/667] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:07.782477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e448b-513b-4883-be55-2364c964293a None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:07.805058 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a7253-887b-464c-9059-e74b3e64e15c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:07.806531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a7253-887b-464c-9059-e74b3e64e15c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:07.883458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e448b-513b-4883-be55-2364c964293a None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:07.913952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:07.922469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fdc9d-9d11-49d0-a4b7-ff1ce82213f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:07.922932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 329/668] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:33:07.923070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fdc9d-9d11-49d0-a4b7-ff1ce82213f8 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:07.923361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fdc9d-9d11-49d0-a4b7-ff1ce82213f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:07.923636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fdc9d-9d11-49d0-a4b7-ff1ce82213f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:07.924773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7437f3-97f3-446c-83b3-33a930eb3ca4 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:07.925386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7437f3-97f3-446c-83b3-33a930eb3ca4 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:07.925756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7437f3-97f3-446c-83b3-33a930eb3ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:07.931443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fdc9d-9d11-49d0-a4b7-ff1ce82213f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:07.933014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fdc9d-9d11-49d0-a4b7-ff1ce82213f8 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-746fdc9d-9d11-49d0-a4b7-ff1ce82213f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:07.944620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:07.945655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:07.946804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:07.946804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:07.946804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:07.946804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:07.946804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:07.946804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:07.946804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:07.946804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:07.947320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-8a7437f3-97f3-446c-83b3-33a930eb3ca4 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:07.948076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8a7437f3-97f3-446c-83b3-33a930eb3ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:07.948326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 340/669] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:07.949278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fdc9d-9d11-49d0-a4b7-ff1ce82213f8 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:08.010812 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:08.018808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 330/670] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:08.019646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7437f3-97f3-446c-83b3-33a930eb3ca4 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:08.056868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fdc9d-9d11-49d0-a4b7-ff1ce82213f8 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:08.090218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5573b6-8279-49b1-958b-35895c7bdaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:08.090958 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5573b6-8279-49b1-958b-35895c7bdaeb None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:08.091332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5573b6-8279-49b1-958b-35895c7bdaeb None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:08.091705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5573b6-8279-49b1-958b-35895c7bdaeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:08.092490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5573b6-8279-49b1-958b-35895c7bdaeb None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=6f224ed840244faea117fe97c29c26d1) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:08.092861 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5573b6-8279-49b1-958b-35895c7bdaeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:08.094391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5573b6-8279-49b1-958b-35895c7bdaeb None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:08.109722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 341/671] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:07 2022] GET /identity/v3/role_assignments?user.id=6f224ed840244faea117fe97c29c26d1&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:08.109722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5573b6-8279-49b1-958b-35895c7bdaeb None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:08.144588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7437f3-97f3-446c-83b3-33a930eb3ca4 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:08.181678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:08.188919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3f8d6-37d6-4388-b405-4d1ab21ed8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:08.189722 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3f8d6-37d6-4388-b405-4d1ab21ed8ae None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:08.189855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3f8d6-37d6-4388-b405-4d1ab21ed8ae None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:08.190077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3f8d6-37d6-4388-b405-4d1ab21ed8ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:08.201766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3f8d6-37d6-4388-b405-4d1ab21ed8ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:08.203421 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3f8d6-37d6-4388-b405-4d1ab21ed8ae None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-44f3f8d6-37d6-4388-b405-4d1ab21ed8ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:08.219888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:08.220962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:08.222000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:08.222000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:08.222000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:08.226425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 331/672] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:08.227877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8c7d7-d454-4a65-a66d-635876fd3255 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:08.228151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8c7d7-d454-4a65-a66d-635876fd3255 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:08.228426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8c7d7-d454-4a65-a66d-635876fd3255 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:08.230298 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 332/673] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:08.231025 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8c7d7-d454-4a65-a66d-635876fd3255 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:08.266811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5573b6-8279-49b1-958b-35895c7bdaeb None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:08.293035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:08.317134 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d606455-e202-4b6c-8740-443ff5b1dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:08.317878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d606455-e202-4b6c-8740-443ff5b1dc0d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:08.318747 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d606455-e202-4b6c-8740-443ff5b1dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:08.319098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d606455-e202-4b6c-8740-443ff5b1dc0d None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:08.319805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d606455-e202-4b6c-8740-443ff5b1dc0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:08.320111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d606455-e202-4b6c-8740-443ff5b1dc0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:08.321554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d606455-e202-4b6c-8740-443ff5b1dc0d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:08.335211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 342/674] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:08.336591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ac379-de77-4583-9069-d207688ae529 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:08.336890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ac379-de77-4583-9069-d207688ae529 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:08.337166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ac379-de77-4583-9069-d207688ae529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:08.353459 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8c7d7-d454-4a65-a66d-635876fd3255 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:08.357104 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-4c8ac379-de77-4583-9069-d207688ae529 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:08.358336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4c8ac379-de77-4583-9069-d207688ae529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:08.391035 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed6a33-9e6d-4776-8ee8-d7690f546e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:08.391651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed6a33-9e6d-4776-8ee8-d7690f546e55 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:08.391937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed6a33-9e6d-4776-8ee8-d7690f546e55 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:08.392209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed6a33-9e6d-4776-8ee8-d7690f546e55 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:08.392970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed6a33-9e6d-4776-8ee8-d7690f546e55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:08.393278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed6a33-9e6d-4776-8ee8-d7690f546e55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:08.394956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed6a33-9e6d-4776-8ee8-d7690f546e55 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:08.407867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 333/675] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:08.408942 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed6a33-9e6d-4776-8ee8-d7690f546e55 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:08.429889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 343/676] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:08.430830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ac379-de77-4583-9069-d207688ae529 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:08.468031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:08.535482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed6a33-9e6d-4776-8ee8-d7690f546e55 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:08.557825 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ac379-de77-4583-9069-d207688ae529 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:08.569385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:08.577683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0559d-de03-4e64-808e-499425a55e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:08.578931 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0559d-de03-4e64-808e-499425a55e58 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:08.579416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0559d-de03-4e64-808e-499425a55e58 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:08.579875 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0559d-de03-4e64-808e-499425a55e58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:08.594480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0559d-de03-4e64-808e-499425a55e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:08.595919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98221d-2be9-4b19-8058-513950a09b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:08.596626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98221d-2be9-4b19-8058-513950a09b44 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:08.597004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98221d-2be9-4b19-8058-513950a09b44 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:08.597376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98221d-2be9-4b19-8058-513950a09b44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:08.598238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98221d-2be9-4b19-8058-513950a09b44 None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=913a866ddc074b7f9aa142c70da98a69) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:08.598701 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98221d-2be9-4b19-8058-513950a09b44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:08.600470 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98221d-2be9-4b19-8058-513950a09b44 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-8db0559d-de03-4e64-808e-499425a55e58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:08.600777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:08.601888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:08.602934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:08.602934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:08.602934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:08.602934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:08.602934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:08.602934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:08.602934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:08.602934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:08.602934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:08.602934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:08.602934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:33:08.602934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:08.610594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 334/677] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:33:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:08.610747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0559d-de03-4e64-808e-499425a55e58 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:08.614882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 344/678] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:08 2022] GET /identity/v3/role_assignments?user.id=913a866ddc074b7f9aa142c70da98a69&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:08.616414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98221d-2be9-4b19-8058-513950a09b44 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:08.772350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0559d-de03-4e64-808e-499425a55e58 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:08.774272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:08.778993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98221d-2be9-4b19-8058-513950a09b44 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:08.812393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3025ce-05d4-4b33-af84-8f6311ec31b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:08.813038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3025ce-05d4-4b33-af84-8f6311ec31b5 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:08.813315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3025ce-05d4-4b33-af84-8f6311ec31b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:08.813592 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3025ce-05d4-4b33-af84-8f6311ec31b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:08.822506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3025ce-05d4-4b33-af84-8f6311ec31b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:08.823962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3025ce-05d4-4b33-af84-8f6311ec31b5 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:08.827790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732970cb-61cc-49f6-bd7c-2615437f462c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:08.828503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732970cb-61cc-49f6-bd7c-2615437f462c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:08.828677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732970cb-61cc-49f6-bd7c-2615437f462c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:08.828932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732970cb-61cc-49f6-bd7c-2615437f462c None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:08.830159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732970cb-61cc-49f6-bd7c-2615437f462c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:08.830159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732970cb-61cc-49f6-bd7c-2615437f462c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:08.831873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732970cb-61cc-49f6-bd7c-2615437f462c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-af3025ce-05d4-4b33-af84-8f6311ec31b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:08.838598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:08.839518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:08.840417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:08.840417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:08.840417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:08.840417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:08.840417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:08.840417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:08.840417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:08.840417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:08.840417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 335/679] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:08.840925 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3025ce-05d4-4b33-af84-8f6311ec31b5 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:08.843747 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 345/680] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:33:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:08.845381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732970cb-61cc-49f6-bd7c-2615437f462c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:08.889195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:08.966625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3025ce-05d4-4b33-af84-8f6311ec31b5 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:08.967244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732970cb-61cc-49f6-bd7c-2615437f462c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:09.008610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2782-9494-4901-822a-68cd17772ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:09.009328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587fba5-504f-474a-9fa4-f8f0e973da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:09.009907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2782-9494-4901-822a-68cd17772ed6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:09.010015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587fba5-504f-474a-9fa4-f8f0e973da05 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:09.010244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587fba5-504f-474a-9fa4-f8f0e973da05 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:09.010651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587fba5-504f-474a-9fa4-f8f0e973da05 None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:09.010883 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2782-9494-4901-822a-68cd17772ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:09.011335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2782-9494-4901-822a-68cd17772ed6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:09.011524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587fba5-504f-474a-9fa4-f8f0e973da05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:09.011897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587fba5-504f-474a-9fa4-f8f0e973da05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:09.013395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587fba5-504f-474a-9fa4-f8f0e973da05 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:09.022622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2782-9494-4901-822a-68cd17772ed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-cf1d2782-9494-4901-822a-68cd17772ed6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:09.027794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:09.028990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:09.035471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:09.035471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:09.035471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:09.035471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:09.035471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:09.035471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:09.035471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:09.035471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:09.035471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:09.035471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:09.035471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:09.035471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:09.036326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 346/681] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:09.036326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587fba5-504f-474a-9fa4-f8f0e973da05 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:09.038017 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 336/682] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:33:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:09.038277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2782-9494-4901-822a-68cd17772ed6 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:09.070394 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:09.134405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:09.138652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587fba5-504f-474a-9fa4-f8f0e973da05 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:09.159899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2782-9494-4901-822a-68cd17772ed6 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:09.174563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cdbc6-6cc7-4d26-9bc0-a25186538dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:09.175468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cdbc6-6cc7-4d26-9bc0-a25186538dff None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:09.175966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cdbc6-6cc7-4d26-9bc0-a25186538dff None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:09.176441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cdbc6-6cc7-4d26-9bc0-a25186538dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:09.187197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cdbc6-6cc7-4d26-9bc0-a25186538dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:09.188864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cdbc6-6cc7-4d26-9bc0-a25186538dff None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:09.199764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848d45b-c4fe-4b58-bdb0-36b3a4fc84ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:09.200366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848d45b-c4fe-4b58-bdb0-36b3a4fc84ba None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:09.200647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848d45b-c4fe-4b58-bdb0-36b3a4fc84ba None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:09.200923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848d45b-c4fe-4b58-bdb0-36b3a4fc84ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:09.201648 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848d45b-c4fe-4b58-bdb0-36b3a4fc84ba None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=50d3b75cbc5c4d1686f4d93248e04746) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:09.201988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848d45b-c4fe-4b58-bdb0-36b3a4fc84ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:09.203414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848d45b-c4fe-4b58-bdb0-36b3a4fc84ba None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-407cdbc6-6cc7-4d26-9bc0-a25186538dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:09.205989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:09.206905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:09.207763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:09.207763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:09.207763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:09.213984 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 337/684] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:09 2022] GET /identity/v3/role_assignments?user.id=50d3b75cbc5c4d1686f4d93248e04746&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:09.213984 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848d45b-c4fe-4b58-bdb0-36b3a4fc84ba None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:09.214214 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 347/684] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:09.218911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cdbc6-6cc7-4d26-9bc0-a25186538dff None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:09.317408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:09.336106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848d45b-c4fe-4b58-bdb0-36b3a4fc84ba None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:09.418455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724364e3-ab6f-40cd-b920-9add13ebb45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:09.419255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724364e3-ab6f-40cd-b920-9add13ebb45a None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:09.419982 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724364e3-ab6f-40cd-b920-9add13ebb45a None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:09.419982 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724364e3-ab6f-40cd-b920-9add13ebb45a None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:09.420720 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724364e3-ab6f-40cd-b920-9add13ebb45a None admin] RBAC: Adding query filter params (name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:09.421118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724364e3-ab6f-40cd-b920-9add13ebb45a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:09.422599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724364e3-ab6f-40cd-b920-9add13ebb45a None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:09.434255 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cdbc6-6cc7-4d26-9bc0-a25186538dff None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:09.444291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 338/685] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:33:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:09.449933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724364e3-ab6f-40cd-b920-9add13ebb45a None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:09.475483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479a2f9-bf97-4918-8386-8d8baa005f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:09.476092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479a2f9-bf97-4918-8386-8d8baa005f8b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:09.476363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479a2f9-bf97-4918-8386-8d8baa005f8b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:09.476634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479a2f9-bf97-4918-8386-8d8baa005f8b None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:09.477276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479a2f9-bf97-4918-8386-8d8baa005f8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:09.477574 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479a2f9-bf97-4918-8386-8d8baa005f8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:09.479352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479a2f9-bf97-4918-8386-8d8baa005f8b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:09.497730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 348/686] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:09.503834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479a2f9-bf97-4918-8386-8d8baa005f8b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:09.523448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:09.591107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724364e3-ab6f-40cd-b920-9add13ebb45a None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:09.639915 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9e557-a438-44e7-b1ab-157aad35f251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:09.641145 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9e557-a438-44e7-b1ab-157aad35f251 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:09.641628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9e557-a438-44e7-b1ab-157aad35f251 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:09.642126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9e557-a438-44e7-b1ab-157aad35f251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:09.642371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479a2f9-bf97-4918-8386-8d8baa005f8b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:09.661765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9e557-a438-44e7-b1ab-157aad35f251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:09.664825 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9e557-a438-44e7-b1ab-157aad35f251 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:09.678242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:09.687808 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe922b0-c78c-4fc1-9b08-1ee4f6bf71d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:09.688579 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe922b0-c78c-4fc1-9b08-1ee4f6bf71d5 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:09.688858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe922b0-c78c-4fc1-9b08-1ee4f6bf71d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:09.689128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe922b0-c78c-4fc1-9b08-1ee4f6bf71d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-51a9e557-a438-44e7-b1ab-157aad35f251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:09.689725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:09.690625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:09.691499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:09.691499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:09.691499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:09.694166 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 339/687] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:09.699701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9e557-a438-44e7-b1ab-157aad35f251 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:09.704551 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe922b0-c78c-4fc1-9b08-1ee4f6bf71d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:09.706328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe922b0-c78c-4fc1-9b08-1ee4f6bf71d5 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-0fe922b0-c78c-4fc1-9b08-1ee4f6bf71d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:09.724093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:09.725580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:09.726555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:09.726555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:09.726555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:09.726555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:09.726555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:09.726555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:09.726555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:09.726555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:09.726555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:33:09.726555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:09.726555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 349/688] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 08:33:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:09.734712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe922b0-c78c-4fc1-9b08-1ee4f6bf71d5 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:09.816376 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9e557-a438-44e7-b1ab-157aad35f251 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:09.855707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1e5d9-c04d-4ae0-9a9d-f3c5eeefb849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:09.856238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1e5d9-c04d-4ae0-9a9d-f3c5eeefb849 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:09.857653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1e5d9-c04d-4ae0-9a9d-f3c5eeefb849 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:09.857962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1e5d9-c04d-4ae0-9a9d-f3c5eeefb849 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:09.858663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1e5d9-c04d-4ae0-9a9d-f3c5eeefb849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:09.858970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1e5d9-c04d-4ae0-9a9d-f3c5eeefb849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:09.860326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1e5d9-c04d-4ae0-9a9d-f3c5eeefb849 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:09.883836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 340/689] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:09.889535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1e5d9-c04d-4ae0-9a9d-f3c5eeefb849 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:09.895259 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:09.963709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:09.987850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe922b0-c78c-4fc1-9b08-1ee4f6bf71d5 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:10.007305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1e5d9-c04d-4ae0-9a9d-f3c5eeefb849 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:10.042789 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f793423-6921-4762-83f4-a4f99a9d6e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:10.043420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f793423-6921-4762-83f4-a4f99a9d6e29 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:10.043693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f793423-6921-4762-83f4-a4f99a9d6e29 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:10.043961 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f793423-6921-4762-83f4-a4f99a9d6e29 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:10.051381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f793423-6921-4762-83f4-a4f99a9d6e29 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:10.053362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f793423-6921-4762-83f4-a4f99a9d6e29 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-1f793423-6921-4762-83f4-a4f99a9d6e29 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:10.066129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:10.067117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:10.068135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:10.068135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:10.068135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:10.068135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:10.068135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:10.068135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:10.068135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:10.068135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:10.068135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:33:10.068135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:10.068135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 341/690] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:33:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:10.070507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2db865-418a-4565-ac54-dca5aba38d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:10.071345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2db865-418a-4565-ac54-dca5aba38d65 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:10.071824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2db865-418a-4565-ac54-dca5aba38d65 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:10.072291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2db865-418a-4565-ac54-dca5aba38d65 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:10.073251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2db865-418a-4565-ac54-dca5aba38d65 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:10.073779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2db865-418a-4565-ac54-dca5aba38d65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:10.074634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f793423-6921-4762-83f4-a4f99a9d6e29 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:10.075769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2db865-418a-4565-ac54-dca5aba38d65 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:10.095214 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 350/691] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 08:33:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:10.101075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2db865-418a-4565-ac54-dca5aba38d65 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:10.260261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2db865-418a-4565-ac54-dca5aba38d65 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:10.289019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:10.299841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f793423-6921-4762-83f4-a4f99a9d6e29 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:10.302706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d1d36-0640-4ea1-ba80-c90709028cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:10.303560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d1d36-0640-4ea1-ba80-c90709028cba None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:10.304050 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d1d36-0640-4ea1-ba80-c90709028cba None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:10.304513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d1d36-0640-4ea1-ba80-c90709028cba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:10.350703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:10.391342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9e3b1-16b6-4f82-99a7-7bd45bfe47bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:10.391959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9e3b1-16b6-4f82-99a7-7bd45bfe47bd None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:10.392235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9e3b1-16b6-4f82-99a7-7bd45bfe47bd None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:10.392507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9e3b1-16b6-4f82-99a7-7bd45bfe47bd None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:10.393228 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9e3b1-16b6-4f82-99a7-7bd45bfe47bd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:10.393529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9e3b1-16b6-4f82-99a7-7bd45bfe47bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:10.395018 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9e3b1-16b6-4f82-99a7-7bd45bfe47bd None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:10.398955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d1d36-0640-4ea1-ba80-c90709028cba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:10.400440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d1d36-0640-4ea1-ba80-c90709028cba None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:10.434313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 342/692] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Oct 4 08:33:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:10.442762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9e3b1-16b6-4f82-99a7-7bd45bfe47bd None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:10.573531 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 351/693] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:33:10.579245 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540f550-c2cd-47a1-91f5-3125a2fd72b5 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:10.579532 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540f550-c2cd-47a1-91f5-3125a2fd72b5 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:10.579800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540f550-c2cd-47a1-91f5-3125a2fd72b5 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:10.581165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 352/694] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:10.581741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540f550-c2cd-47a1-91f5-3125a2fd72b5 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:10.587526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9e3b1-16b6-4f82-99a7-7bd45bfe47bd None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:10.626084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:10.630423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d44f3-b7c3-496d-901e-9afc8739ed38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:10.631795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d44f3-b7c3-496d-901e-9afc8739ed38 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:10.632275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d44f3-b7c3-496d-901e-9afc8739ed38 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:10.632763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d44f3-b7c3-496d-901e-9afc8739ed38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:10.700097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:10.713200 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540f550-c2cd-47a1-91f5-3125a2fd72b5 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:10.728184 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d44f3-b7c3-496d-901e-9afc8739ed38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:10.729804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d44f3-b7c3-496d-901e-9afc8739ed38 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:10.749879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5465e-f65a-46bd-af40-5c67a8cd701f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:10.750831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5465e-f65a-46bd-af40-5c67a8cd701f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:10.751304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5465e-f65a-46bd-af40-5c67a8cd701f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:10.751789 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5465e-f65a-46bd-af40-5c67a8cd701f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:10.762993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5465e-f65a-46bd-af40-5c67a8cd701f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:10.764772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb5465e-f65a-46bd-af40-5c67a8cd701f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-6cb5465e-f65a-46bd-af40-5c67a8cd701f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:10.780614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:10.781567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:10.782516 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:10.782516 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:10.782516 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:10.788995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 353/695] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:10.790529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02556e-5032-4de9-ad01-62370a45e557 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:10.791036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02556e-5032-4de9-ad01-62370a45e557 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:10.791503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02556e-5032-4de9-ad01-62370a45e557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:10.810653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-1d02556e-5032-4de9-ad01-62370a45e557 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:10.812132 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1d02556e-5032-4de9-ad01-62370a45e557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:10.846308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 343/696] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 08:33:10.847261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d44f3-b7c3-496d-901e-9afc8739ed38 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:10.885598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 354/697] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:10.886810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02556e-5032-4de9-ad01-62370a45e557 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:10.895471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:11.004339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02556e-5032-4de9-ad01-62370a45e557 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:11.027439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:11.046952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ed1f4-fadd-4fd2-91e1-5ef89bf47333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:11.047881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ed1f4-fadd-4fd2-91e1-5ef89bf47333 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:11.048353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ed1f4-fadd-4fd2-91e1-5ef89bf47333 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:11.048797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ed1f4-fadd-4fd2-91e1-5ef89bf47333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:11.063262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ed1f4-fadd-4fd2-91e1-5ef89bf47333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:11.065369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ed1f4-fadd-4fd2-91e1-5ef89bf47333 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-e14ed1f4-fadd-4fd2-91e1-5ef89bf47333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:11.084793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:11.086052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:11.087143 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:11.087143 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:11.087143 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:11.088977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 355/698] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:11.093913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d44f3-b7c3-496d-901e-9afc8739ed38 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:11.098650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ed1f4-fadd-4fd2-91e1-5ef89bf47333 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:11.166219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:11.187357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0881c-4e33-4f78-82db-aaf56ee17f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:11.188254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0881c-4e33-4f78-82db-aaf56ee17f9c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:11.188687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0881c-4e33-4f78-82db-aaf56ee17f9c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:11.189464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0881c-4e33-4f78-82db-aaf56ee17f9c None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:11.190286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0881c-4e33-4f78-82db-aaf56ee17f9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:11.191050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0881c-4e33-4f78-82db-aaf56ee17f9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:11.192908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0881c-4e33-4f78-82db-aaf56ee17f9c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:11.209895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 344/699] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:11.211476 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62c0b9-c1f7-46a3-ac11-a1742a9e750e None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:11.211939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62c0b9-c1f7-46a3-ac11-a1742a9e750e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:11.212345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62c0b9-c1f7-46a3-ac11-a1742a9e750e None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:11.213975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 345/700] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:11.217819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62c0b9-c1f7-46a3-ac11-a1742a9e750e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:11.222207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ed1f4-fadd-4fd2-91e1-5ef89bf47333 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:11.259303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb606668-ea7f-4476-a8e1-a6c62a3baabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:11.260213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb606668-ea7f-4476-a8e1-a6c62a3baabd None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:11.260493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb606668-ea7f-4476-a8e1-a6c62a3baabd None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:11.260768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb606668-ea7f-4476-a8e1-a6c62a3baabd None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:11.261433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb606668-ea7f-4476-a8e1-a6c62a3baabd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:11.261766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb606668-ea7f-4476-a8e1-a6c62a3baabd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:11.263186 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb606668-ea7f-4476-a8e1-a6c62a3baabd None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:11.278039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 356/701] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:11.278039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69013d04-8495-4cdb-af03-158746949932 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:11.278039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69013d04-8495-4cdb-af03-158746949932 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:11.278389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69013d04-8495-4cdb-af03-158746949932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:11.297370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-69013d04-8495-4cdb-af03-158746949932 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:11.298594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-69013d04-8495-4cdb-af03-158746949932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:11.333457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62c0b9-c1f7-46a3-ac11-a1742a9e750e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:11.370122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf231b55-c7e1-441e-9b4c-ebc222c82618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:11.381884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf231b55-c7e1-441e-9b4c-ebc222c82618 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:11.382029 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 357/702] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:11.382110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf231b55-c7e1-441e-9b4c-ebc222c82618 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:11.382110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf231b55-c7e1-441e-9b4c-ebc222c82618 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:11.382263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69013d04-8495-4cdb-af03-158746949932 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:11.390312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf231b55-c7e1-441e-9b4c-ebc222c82618 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:11.392299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf231b55-c7e1-441e-9b4c-ebc222c82618 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:11.394216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-bf231b55-c7e1-441e-9b4c-ebc222c82618 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:11.407601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:11.408787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:11.410012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:11.410012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:11.410012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:11.410012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:11.410012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:11.410012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:11.410012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:11.410012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:11.413624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 346/703] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:11.415315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf231b55-c7e1-441e-9b4c-ebc222c82618 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:11.433484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:11.498942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69013d04-8495-4cdb-af03-158746949932 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:11.532475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf231b55-c7e1-441e-9b4c-ebc222c82618 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:11.545633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c599701-8708-49fe-abc7-f99da77f32eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:11.546892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c599701-8708-49fe-abc7-f99da77f32eb None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:11.547845 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c599701-8708-49fe-abc7-f99da77f32eb None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:11.548464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c599701-8708-49fe-abc7-f99da77f32eb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:11.549331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c599701-8708-49fe-abc7-f99da77f32eb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:11.551582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c599701-8708-49fe-abc7-f99da77f32eb None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-1c599701-8708-49fe-abc7-f99da77f32eb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:11.566858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:11.567896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:33:11.569316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:11.570089 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe07e4-a17f-4696-bcfd-c83fba84f2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:11.570089 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe07e4-a17f-4696-bcfd-c83fba84f2fd None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:11.570234 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe07e4-a17f-4696-bcfd-c83fba84f2fd None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:11.570392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe07e4-a17f-4696-bcfd-c83fba84f2fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:11.571908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 358/704] 10.222.0.26 () {56 vars in 1129 bytes} [Tue Oct 4 08:33:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:11.573224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c599701-8708-49fe-abc7-f99da77f32eb None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:11.579534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe07e4-a17f-4696-bcfd-c83fba84f2fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:11.581116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe07e4-a17f-4696-bcfd-c83fba84f2fd None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-3abe07e4-a17f-4696-bcfd-c83fba84f2fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:11.595174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:11.596111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:11.596991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:11.596991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:11.596991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:11.596991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:11.596991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:11.596991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:11.596991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:11.596991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:11.596991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 347/705] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:11.598105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe07e4-a17f-4696-bcfd-c83fba84f2fd None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:11.682084 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:11.711109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c599701-8708-49fe-abc7-f99da77f32eb None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:11.725766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:11.730844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe07e4-a17f-4696-bcfd-c83fba84f2fd None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:11.750774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f141f6-0b29-40d0-be9d-8bcb9e90358e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:11.751382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f141f6-0b29-40d0-be9d-8bcb9e90358e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:11.751664 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f141f6-0b29-40d0-be9d-8bcb9e90358e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:11.752215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f141f6-0b29-40d0-be9d-8bcb9e90358e None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:11.752623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f141f6-0b29-40d0-be9d-8bcb9e90358e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:11.752974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f141f6-0b29-40d0-be9d-8bcb9e90358e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:11.754463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f141f6-0b29-40d0-be9d-8bcb9e90358e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:11.772396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 359/706] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:11.772396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f141f6-0b29-40d0-be9d-8bcb9e90358e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:11.773982 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df30da-3b2b-4b0b-9c2e-a1483f85a453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:11.774596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df30da-3b2b-4b0b-9c2e-a1483f85a453 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:11.774881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df30da-3b2b-4b0b-9c2e-a1483f85a453 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:11.775153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df30da-3b2b-4b0b-9c2e-a1483f85a453 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:11.776587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df30da-3b2b-4b0b-9c2e-a1483f85a453 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:11.776587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df30da-3b2b-4b0b-9c2e-a1483f85a453 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:11.777443 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df30da-3b2b-4b0b-9c2e-a1483f85a453 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:11.786701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 348/707] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Oct 4 08:33:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:11.786701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df30da-3b2b-4b0b-9c2e-a1483f85a453 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:11.885712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df30da-3b2b-4b0b-9c2e-a1483f85a453 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:11.896270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f141f6-0b29-40d0-be9d-8bcb9e90358e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:11.907900 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:11.921051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb995ce6-1c2d-4a0a-aa0e-d17550ce7482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:11.921674 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb995ce6-1c2d-4a0a-aa0e-d17550ce7482 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:11.921976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb995ce6-1c2d-4a0a-aa0e-d17550ce7482 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:11.922248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb995ce6-1c2d-4a0a-aa0e-d17550ce7482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:11.932984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629ad9a-5a13-43cf-bcc8-b284d598dc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:11.933240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb995ce6-1c2d-4a0a-aa0e-d17550ce7482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:11.933580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629ad9a-5a13-43cf-bcc8-b284d598dc1d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:11.933894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629ad9a-5a13-43cf-bcc8-b284d598dc1d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:11.934163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629ad9a-5a13-43cf-bcc8-b284d598dc1d None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:11.934704 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb995ce6-1c2d-4a0a-aa0e-d17550ce7482 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:11.934847 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629ad9a-5a13-43cf-bcc8-b284d598dc1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:11.935155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629ad9a-5a13-43cf-bcc8-b284d598dc1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:11.936562 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629ad9a-5a13-43cf-bcc8-b284d598dc1d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:11.948436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 360/708] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:11.949208 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629ad9a-5a13-43cf-bcc8-b284d598dc1d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-eb995ce6-1c2d-4a0a-aa0e-d17550ce7482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:11.950324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:11.951277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:11.952708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:11.952708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:11.952708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:11.952708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 349/709] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:11.952708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb995ce6-1c2d-4a0a-aa0e-d17550ce7482 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:11.984513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:12.051435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb995ce6-1c2d-4a0a-aa0e-d17550ce7482 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:12.051639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629ad9a-5a13-43cf-bcc8-b284d598dc1d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:12.083597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0b5b2-06c2-4dd6-a7b8-4077048c68d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:12.084751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0b5b2-06c2-4dd6-a7b8-4077048c68d2 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:12.084923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f88fe-7881-41e7-a4ce-fceeeb380a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:12.085045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0b5b2-06c2-4dd6-a7b8-4077048c68d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:12.085283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0b5b2-06c2-4dd6-a7b8-4077048c68d2 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:12.085522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f88fe-7881-41e7-a4ce-fceeeb380a4e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:12.085836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f88fe-7881-41e7-a4ce-fceeeb380a4e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:12.085928 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0b5b2-06c2-4dd6-a7b8-4077048c68d2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:12.086115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f88fe-7881-41e7-a4ce-fceeeb380a4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:12.086203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0b5b2-06c2-4dd6-a7b8-4077048c68d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:12.086831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f88fe-7881-41e7-a4ce-fceeeb380a4e None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=ee8a19f4d9704408a8b737095e5885f1) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:12.087143 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f88fe-7881-41e7-a4ce-fceeeb380a4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:12.087563 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0b5b2-06c2-4dd6-a7b8-4077048c68d2 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:12.088562 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f88fe-7881-41e7-a4ce-fceeeb380a4e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:12.092912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 350/710] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Oct 4 08:33:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:12.106166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 361/711] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:11 2022] GET /identity/v3/role_assignments?user.id=ee8a19f4d9704408a8b737095e5885f1&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:12.106380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd8b02-b834-440a-8a91-7582a8cc5182 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:12.106380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd8b02-b834-440a-8a91-7582a8cc5182 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:12.106380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd8b02-b834-440a-8a91-7582a8cc5182 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:12.106575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f88fe-7881-41e7-a4ce-fceeeb380a4e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:12.106676 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 351/712] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:12.106676 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd8b02-b834-440a-8a91-7582a8cc5182 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:12.174981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:12.249611 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:12.260109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f88fe-7881-41e7-a4ce-fceeeb380a4e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:12.260490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd8b02-b834-440a-8a91-7582a8cc5182 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:12.306815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5baf7c-113e-4ff6-8793-fe862ef8d2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:12.307053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6002ea-5059-4a33-8430-35abdab07c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:12.307605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6002ea-5059-4a33-8430-35abdab07c6e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:12.307887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6002ea-5059-4a33-8430-35abdab07c6e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:12.308134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5baf7c-113e-4ff6-8793-fe862ef8d2b4 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:12.308246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6002ea-5059-4a33-8430-35abdab07c6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:12.308647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5baf7c-113e-4ff6-8793-fe862ef8d2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:12.308909 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6002ea-5059-4a33-8430-35abdab07c6e None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=b232b71c77a24be5bfcdb6aeb5807f90) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:12.309216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5baf7c-113e-4ff6-8793-fe862ef8d2b4 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:12.309339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6002ea-5059-4a33-8430-35abdab07c6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:12.310226 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5baf7c-113e-4ff6-8793-fe862ef8d2b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:12.310673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5baf7c-113e-4ff6-8793-fe862ef8d2b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:12.310799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6002ea-5059-4a33-8430-35abdab07c6e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:12.313026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5baf7c-113e-4ff6-8793-fe862ef8d2b4 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:12.317253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 362/713] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:12 2022] GET /identity/v3/role_assignments?user.id=b232b71c77a24be5bfcdb6aeb5807f90&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:12.319288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d802b6-f809-4d69-97dd-4e8c7d10aec8 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:12.319646 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d802b6-f809-4d69-97dd-4e8c7d10aec8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:12.319993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d802b6-f809-4d69-97dd-4e8c7d10aec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:12.329556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 352/714] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:12.338193 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5baf7c-113e-4ff6-8793-fe862ef8d2b4 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:12.462900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-87d802b6-f809-4d69-97dd-4e8c7d10aec8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:12.465327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-87d802b6-f809-4d69-97dd-4e8c7d10aec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:12.510959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:12.518471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5baf7c-113e-4ff6-8793-fe862ef8d2b4 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:12.572700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672737f-8003-441c-b51e-75dc80dbd60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:12.574106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672737f-8003-441c-b51e-75dc80dbd60a None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:12.574570 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672737f-8003-441c-b51e-75dc80dbd60a None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:12.575023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672737f-8003-441c-b51e-75dc80dbd60a None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:12.575992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672737f-8003-441c-b51e-75dc80dbd60a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:12.577811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672737f-8003-441c-b51e-75dc80dbd60a None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:12.581711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 363/715] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:12.607958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 353/716] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:33:12 2022] POST /identity/v3/users => generated 254 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:33:12.863850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d802b6-f809-4d69-97dd-4e8c7d10aec8 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:12.972810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:12.985104 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d802b6-f809-4d69-97dd-4e8c7d10aec8 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:13.036284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80d5ee-a371-4e9f-b11f-f454e1d6b4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:13.037004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80d5ee-a371-4e9f-b11f-f454e1d6b4a9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:13.037284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80d5ee-a371-4e9f-b11f-f454e1d6b4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:13.037557 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80d5ee-a371-4e9f-b11f-f454e1d6b4a9 None admin] PATH_INFO: `/v3/services/network` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:13.038168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80d5ee-a371-4e9f-b11f-f454e1d6b4a9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:13.039545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80d5ee-a371-4e9f-b11f-f454e1d6b4a9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-ad80d5ee-a371-4e9f-b11f-f454e1d6b4a9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:13.050704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:13.051744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:13.053526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 364/717] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:12 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:13.060899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672737f-8003-441c-b51e-75dc80dbd60a None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:13.234521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672737f-8003-441c-b51e-75dc80dbd60a None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:13.301347 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a66a9c-c54d-415f-a9b2-54f374202638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:13.302420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a66a9c-c54d-415f-a9b2-54f374202638 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:13.302953 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a66a9c-c54d-415f-a9b2-54f374202638 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:13.303430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a66a9c-c54d-415f-a9b2-54f374202638 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:13.304275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a66a9c-c54d-415f-a9b2-54f374202638 None admin] RBAC: Adding query filter params (name=network) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:13.304791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a66a9c-c54d-415f-a9b2-54f374202638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:13.306388 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a66a9c-c54d-415f-a9b2-54f374202638 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:13.327390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 354/718] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:13.338766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80d5ee-a371-4e9f-b11f-f454e1d6b4a9 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:13.470386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80d5ee-a371-4e9f-b11f-f454e1d6b4a9 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:13.523042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b01711-9e6e-46ca-b78c-134f0336118f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:13.523669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b01711-9e6e-46ca-b78c-134f0336118f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:13.523949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b01711-9e6e-46ca-b78c-134f0336118f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:13.524221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b01711-9e6e-46ca-b78c-134f0336118f None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:13.524861 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b01711-9e6e-46ca-b78c-134f0336118f None admin] RBAC: Adding query filter params (type=network) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:13.525169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b01711-9e6e-46ca-b78c-134f0336118f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:13.526569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b01711-9e6e-46ca-b78c-134f0336118f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:13.538726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 365/719] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:13 2022] GET /identity/v3/services?type=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:14.504294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a74ac-8103-4eeb-aa62-beab44e9a585 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:14.504814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a74ac-8103-4eeb-aa62-beab44e9a585 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:14.504921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a74ac-8103-4eeb-aa62-beab44e9a585 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:14.506549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 355/720] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:14.515884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be52c69-ac1f-49f1-84db-134fca305a2e None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:14.516726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be52c69-ac1f-49f1-84db-134fca305a2e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:14.517206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be52c69-ac1f-49f1-84db-134fca305a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:14.546739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-2be52c69-ac1f-49f1-84db-134fca305a2e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:14.548292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2be52c69-ac1f-49f1-84db-134fca305a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:14.562896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:14.637561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 366/721] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:14.847717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485cf09-3183-4e95-8c52-6ea363925b89 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:14.848415 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485cf09-3183-4e95-8c52-6ea363925b89 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:14.848983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485cf09-3183-4e95-8c52-6ea363925b89 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:14.851312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 356/722] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:14.856813 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4db42-f053-4aa4-a6ab-c39b82e03f3c None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:14.857324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4db42-f053-4aa4-a6ab-c39b82e03f3c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:14.857827 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4db42-f053-4aa4-a6ab-c39b82e03f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:14.876789 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-69a4db42-f053-4aa4-a6ab-c39b82e03f3c None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:14.878315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-69a4db42-f053-4aa4-a6ab-c39b82e03f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:14.956304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 367/723] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:14.978854 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485cf09-3183-4e95-8c52-6ea363925b89 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:15.000731 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:15.151125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485cf09-3183-4e95-8c52-6ea363925b89 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:15.196744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c9a4b-4daf-4f45-8711-56c2dc5502b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:15.197597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c9a4b-4daf-4f45-8711-56c2dc5502b9 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:15.198050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c9a4b-4daf-4f45-8711-56c2dc5502b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:15.198535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c9a4b-4daf-4f45-8711-56c2dc5502b9 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:15.199526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c9a4b-4daf-4f45-8711-56c2dc5502b9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:15.201035 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c9a4b-4daf-4f45-8711-56c2dc5502b9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:15.226148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 357/724] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:33:14 2022] POST /identity/v3/services => generated 251 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:33:15.258988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4db42-f053-4aa4-a6ab-c39b82e03f3c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:15.352510 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:15.388590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4db42-f053-4aa4-a6ab-c39b82e03f3c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:15.455322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb91ef-804b-4365-9c65-3bb4cc5ee586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:15.455870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb91ef-804b-4365-9c65-3bb4cc5ee586 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:15.456144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb91ef-804b-4365-9c65-3bb4cc5ee586 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:15.456412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb91ef-804b-4365-9c65-3bb4cc5ee586 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:15.470232 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb91ef-804b-4365-9c65-3bb4cc5ee586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-bdeb91ef-804b-4365-9c65-3bb4cc5ee586 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:15.475565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:15.476527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:15.477779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 368/725] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:33:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:15.485023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c9a4b-4daf-4f45-8711-56c2dc5502b9 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:15.517723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ce75b-93bf-45c4-be3e-c61447b2b896 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:15.518344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ce75b-93bf-45c4-be3e-c61447b2b896 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:15.518344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ce75b-93bf-45c4-be3e-c61447b2b896 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:15.520035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 369/726] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:15.528222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6016a3-64a7-4d91-92b0-3bed7a796e14 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:15.528504 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6016a3-64a7-4d91-92b0-3bed7a796e14 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:15.528775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6016a3-64a7-4d91-92b0-3bed7a796e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:15.556770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-3c6016a3-64a7-4d91-92b0-3bed7a796e14 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:15.558554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3c6016a3-64a7-4d91-92b0-3bed7a796e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:15.589871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:15.624591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c9a4b-4daf-4f45-8711-56c2dc5502b9 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:15.664377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 370/727] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 08:33:15.681413 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff68bee-ed08-43c0-a377-07fef29d9b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:15.682059 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff68bee-ed08-43c0-a377-07fef29d9b53 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:15.682343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff68bee-ed08-43c0-a377-07fef29d9b53 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:15.682622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff68bee-ed08-43c0-a377-07fef29d9b53 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:15.684422 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff68bee-ed08-43c0-a377-07fef29d9b53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:15.684734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff68bee-ed08-43c0-a377-07fef29d9b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:15.686185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff68bee-ed08-43c0-a377-07fef29d9b53 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:15.700068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 358/728] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:33:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:15.702654 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6016a3-64a7-4d91-92b0-3bed7a796e14 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:15.713236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37a5de-0699-4e9d-9ddb-27116238aa82 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:15.713500 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37a5de-0699-4e9d-9ddb-27116238aa82 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:15.713873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37a5de-0699-4e9d-9ddb-27116238aa82 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:15.715572 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 359/729] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:15.721666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8ec83-df35-4373-b16e-e0c1229ddbcf None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:15.721974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8ec83-df35-4373-b16e-e0c1229ddbcf None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:15.722350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8ec83-df35-4373-b16e-e0c1229ddbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:15.758488 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-36c8ec83-df35-4373-b16e-e0c1229ddbcf None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:15.759752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-36c8ec83-df35-4373-b16e-e0c1229ddbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:15.804994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:15.875443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6016a3-64a7-4d91-92b0-3bed7a796e14 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:15.895047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 360/730] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:15.915176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3064fbd-70cc-4b1f-9305-1496fdbc44f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:15.915806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3064fbd-70cc-4b1f-9305-1496fdbc44f4 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:15.916087 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3064fbd-70cc-4b1f-9305-1496fdbc44f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:15.916360 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3064fbd-70cc-4b1f-9305-1496fdbc44f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:15.926796 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3064fbd-70cc-4b1f-9305-1496fdbc44f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:15.928789 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3064fbd-70cc-4b1f-9305-1496fdbc44f4 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-e3064fbd-70cc-4b1f-9305-1496fdbc44f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:15.943624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:15.944594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:15.945472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:15.945472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:15.945472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:15.945472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 371/731] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:15.950641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3064fbd-70cc-4b1f-9305-1496fdbc44f4 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:15.950988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8ec83-df35-4373-b16e-e0c1229ddbcf None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:16.062127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:16.111939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8ec83-df35-4373-b16e-e0c1229ddbcf None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:16.126472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3064fbd-70cc-4b1f-9305-1496fdbc44f4 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:16.157676 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8977e-bf67-4288-99df-126fac697015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:16.158328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8977e-bf67-4288-99df-126fac697015 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:16.158605 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8977e-bf67-4288-99df-126fac697015 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:16.158871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8977e-bf67-4288-99df-126fac697015 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:16.165129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:16.178980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8977e-bf67-4288-99df-126fac697015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-2ca8977e-bf67-4288-99df-126fac697015 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:16.184600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:16.185545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:16.186439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 361/732] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:33:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:16.187675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6e407-c86f-4bdd-8d07-fc61efc42f80 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:16.187675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6e407-c86f-4bdd-8d07-fc61efc42f80 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:16.187832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6e407-c86f-4bdd-8d07-fc61efc42f80 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:16.190340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 362/733] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:16.193743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6e407-c86f-4bdd-8d07-fc61efc42f80 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:16.202655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302a733-905f-4587-ac7e-27b97ba51b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:16.203546 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302a733-905f-4587-ac7e-27b97ba51b59 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:16.204050 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302a733-905f-4587-ac7e-27b97ba51b59 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:16.204435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302a733-905f-4587-ac7e-27b97ba51b59 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:16.205185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302a733-905f-4587-ac7e-27b97ba51b59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:16.205582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302a733-905f-4587-ac7e-27b97ba51b59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:16.207072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302a733-905f-4587-ac7e-27b97ba51b59 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:16.223002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 372/734] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:16.223002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1981ad-e5a9-47db-966a-0e9f759790bf None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:16.223002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1981ad-e5a9-47db-966a-0e9f759790bf None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:16.223002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1981ad-e5a9-47db-966a-0e9f759790bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:16.239279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-fb1981ad-e5a9-47db-966a-0e9f759790bf None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:16.240999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fb1981ad-e5a9-47db-966a-0e9f759790bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:16.309449 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 373/735] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:16 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 08:33:16.310334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1981ad-e5a9-47db-966a-0e9f759790bf None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:16.319332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6e407-c86f-4bdd-8d07-fc61efc42f80 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:16.353768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:16.362540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66547a-cd32-4ee8-b32b-5fb110c2ad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:16.363410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66547a-cd32-4ee8-b32b-5fb110c2ad2b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:16.363901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66547a-cd32-4ee8-b32b-5fb110c2ad2b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:16.364466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66547a-cd32-4ee8-b32b-5fb110c2ad2b None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:16.365969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66547a-cd32-4ee8-b32b-5fb110c2ad2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:16.366508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66547a-cd32-4ee8-b32b-5fb110c2ad2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:16.368792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66547a-cd32-4ee8-b32b-5fb110c2ad2b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:16.379062 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 363/736] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:33:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:16.380112 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66547a-cd32-4ee8-b32b-5fb110c2ad2b None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:16.416678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1981ad-e5a9-47db-966a-0e9f759790bf None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:16.425657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:16.455716 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764df5a4-a947-46b0-9304-0c9e3e50136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:16.456348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764df5a4-a947-46b0-9304-0c9e3e50136f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:16.456629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764df5a4-a947-46b0-9304-0c9e3e50136f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:16.456957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764df5a4-a947-46b0-9304-0c9e3e50136f None admin] PATH_INFO: `/v3/users/nova` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:16.465007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764df5a4-a947-46b0-9304-0c9e3e50136f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:16.466610 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764df5a4-a947-46b0-9304-0c9e3e50136f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-764df5a4-a947-46b0-9304-0c9e3e50136f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:16.479599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:16.480537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:16.481464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:16.481464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:16.481464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:16.481464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:16.481464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:16.481464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:16.481464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:16.481464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:16.481464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:33:16.481464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:16.485530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 374/737] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 08:33:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:16.486423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764df5a4-a947-46b0-9304-0c9e3e50136f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:16.517730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66547a-cd32-4ee8-b32b-5fb110c2ad2b None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:16.553611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c2fd9-3050-41f3-bc1c-15296aa770dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:16.554274 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c2fd9-3050-41f3-bc1c-15296aa770dc None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:16.554558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c2fd9-3050-41f3-bc1c-15296aa770dc None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:16.554840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c2fd9-3050-41f3-bc1c-15296aa770dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:16.565652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c2fd9-3050-41f3-bc1c-15296aa770dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-991c2fd9-3050-41f3-bc1c-15296aa770dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:16.571453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:16.572450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:16.573281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:16.573281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:16.573281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:16.573281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:16.573281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:16.573281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:16.573281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:16.573281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:16.573281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:16.573281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:16.573281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:16.573281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:16.577381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 364/738] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:33:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:16.578326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c2fd9-3050-41f3-bc1c-15296aa770dc None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:16.603056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764df5a4-a947-46b0-9304-0c9e3e50136f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:16.633899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:16.638705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d203e0-509e-4381-894b-f5e5bd4d43c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:16.639364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d203e0-509e-4381-894b-f5e5bd4d43c9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:16.639645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d203e0-509e-4381-894b-f5e5bd4d43c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:16.639916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d203e0-509e-4381-894b-f5e5bd4d43c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:16.650805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d203e0-509e-4381-894b-f5e5bd4d43c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:16.652259 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d203e0-509e-4381-894b-f5e5bd4d43c9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-a2d203e0-509e-4381-894b-f5e5bd4d43c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:16.673583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:16.674621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:16.675511 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:16.675511 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:16.675511 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:16.679719 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 375/739] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:16.679719 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d203e0-509e-4381-894b-f5e5bd4d43c9 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:16.715043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c2fd9-3050-41f3-bc1c-15296aa770dc None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:16.716983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:16.755698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327753e-84d1-4040-9e31-50007a748bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:16.756509 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327753e-84d1-4040-9e31-50007a748bb8 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:16.756859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327753e-84d1-4040-9e31-50007a748bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:16.757202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327753e-84d1-4040-9e31-50007a748bb8 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:16.758049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327753e-84d1-4040-9e31-50007a748bb8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:16.758421 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327753e-84d1-4040-9e31-50007a748bb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:16.760041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327753e-84d1-4040-9e31-50007a748bb8 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:16.779705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 365/740] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 08:33:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:16.780693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327753e-84d1-4040-9e31-50007a748bb8 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:16.818085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d203e0-509e-4381-894b-f5e5bd4d43c9 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:16.858978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f57ca6-5487-4b9f-aab0-3ab4aa162386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:16.859505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f57ca6-5487-4b9f-aab0-3ab4aa162386 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:16.860117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f57ca6-5487-4b9f-aab0-3ab4aa162386 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:16.860117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f57ca6-5487-4b9f-aab0-3ab4aa162386 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:16.860707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f57ca6-5487-4b9f-aab0-3ab4aa162386 None admin] RBAC: Adding query filter params (name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:16.861011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f57ca6-5487-4b9f-aab0-3ab4aa162386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:16.862768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f57ca6-5487-4b9f-aab0-3ab4aa162386 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:16.877296 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 376/741] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:33:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:16.877296 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f57ca6-5487-4b9f-aab0-3ab4aa162386 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:16.902915 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327753e-84d1-4040-9e31-50007a748bb8 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:16.914976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:16.944281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfa2c4-998e-47c5-95b6-05d006807abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:16.945160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfa2c4-998e-47c5-95b6-05d006807abf None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:16.945514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfa2c4-998e-47c5-95b6-05d006807abf None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:16.946176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bfa2c4-998e-47c5-95b6-05d006807abf None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:16.947171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfa2c4-998e-47c5-95b6-05d006807abf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:16.948850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bfa2c4-998e-47c5-95b6-05d006807abf None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:16.963921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 366/742] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:33:16 2022] POST /identity/v3/services => generated 240 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:33:16.964855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfa2c4-998e-47c5-95b6-05d006807abf None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:16.978313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:17.036430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f57ca6-5487-4b9f-aab0-3ab4aa162386 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:17.138858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a0e6b-b328-426a-9e9b-97a85367e371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:17.139781 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a0e6b-b328-426a-9e9b-97a85367e371 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:17.140287 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a0e6b-b328-426a-9e9b-97a85367e371 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:17.140755 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a0e6b-b328-426a-9e9b-97a85367e371 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:17.141579 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a0e6b-b328-426a-9e9b-97a85367e371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:17.142098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a0e6b-b328-426a-9e9b-97a85367e371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:17.143669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a0e6b-b328-426a-9e9b-97a85367e371 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:17.164774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 377/743] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:17.165931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a0e6b-b328-426a-9e9b-97a85367e371 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:17.184688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bfa2c4-998e-47c5-95b6-05d006807abf None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:17.228754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf8427-4326-4ed9-a3b1-8329a3411c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:17.229951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf8427-4326-4ed9-a3b1-8329a3411c2c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:17.230322 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf8427-4326-4ed9-a3b1-8329a3411c2c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:17.230657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf8427-4326-4ed9-a3b1-8329a3411c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:17.245098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf8427-4326-4ed9-a3b1-8329a3411c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:17.246736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf8427-4326-4ed9-a3b1-8329a3411c2c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-29bf8427-4326-4ed9-a3b1-8329a3411c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:17.265908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:17.266893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:17.267765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:17.267765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:17.267765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:17.267765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 367/744] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:17.268073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf8427-4326-4ed9-a3b1-8329a3411c2c None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:17.299516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:17.304568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a0e6b-b328-426a-9e9b-97a85367e371 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:17.355116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badfa7b-32ba-4115-a362-924211bf51ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:17.355752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badfa7b-32ba-4115-a362-924211bf51ce None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:17.356032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badfa7b-32ba-4115-a362-924211bf51ce None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:17.356309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badfa7b-32ba-4115-a362-924211bf51ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:17.361079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:17.373325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badfa7b-32ba-4115-a362-924211bf51ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:17.375842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badfa7b-32ba-4115-a362-924211bf51ce None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:17.386899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf8427-4326-4ed9-a3b1-8329a3411c2c None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-0badfa7b-32ba-4115-a362-924211bf51ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:17.398297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:17.399256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:17.400133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:17.400133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:17.400133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:17.402453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 378/745] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:17.403467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badfa7b-32ba-4115-a362-924211bf51ce None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:17.433817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5575c-f532-4304-b8e3-3729751a4a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:17.434895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5575c-f532-4304-b8e3-3729751a4a19 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:17.435316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5575c-f532-4304-b8e3-3729751a4a19 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:17.435759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5575c-f532-4304-b8e3-3729751a4a19 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:17.445721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5575c-f532-4304-b8e3-3729751a4a19 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:17.447370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5575c-f532-4304-b8e3-3729751a4a19 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-88b5575c-f532-4304-b8e3-3729751a4a19 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:17.460813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:17.462096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:17.463086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:17.463086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:17.463086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:17.463086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:17.463086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:17.463086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:17.463086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:17.463086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:17.463086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:33:17.463086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:17.465623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 368/746] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:33:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:17.466686 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5575c-f532-4304-b8e3-3729751a4a19 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:17.520317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badfa7b-32ba-4115-a362-924211bf51ce None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:17.556357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c23710-ca55-4361-9f00-6cab95388018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:17.557334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c23710-ca55-4361-9f00-6cab95388018 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:17.557722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c23710-ca55-4361-9f00-6cab95388018 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:17.558120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c23710-ca55-4361-9f00-6cab95388018 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:17.559787 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c23710-ca55-4361-9f00-6cab95388018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:17.559787 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c23710-ca55-4361-9f00-6cab95388018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:17.561292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c23710-ca55-4361-9f00-6cab95388018 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:17.579434 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 379/747] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:17.580457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c23710-ca55-4361-9f00-6cab95388018 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:17.586299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:17.607376 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5575c-f532-4304-b8e3-3729751a4a19 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:17.646092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6665e7c-1f55-4a15-a9b9-6f6b8a9f94ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:17.647405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6665e7c-1f55-4a15-a9b9-6f6b8a9f94ce None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:17.647842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6665e7c-1f55-4a15-a9b9-6f6b8a9f94ce None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:17.648483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6665e7c-1f55-4a15-a9b9-6f6b8a9f94ce None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:17.650388 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6665e7c-1f55-4a15-a9b9-6f6b8a9f94ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:17.650852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6665e7c-1f55-4a15-a9b9-6f6b8a9f94ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:17.654037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:17.654441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6665e7c-1f55-4a15-a9b9-6f6b8a9f94ce None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:17.672507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 369/748] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:17.672507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6665e7c-1f55-4a15-a9b9-6f6b8a9f94ce None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:17.706603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c23710-ca55-4361-9f00-6cab95388018 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:17.747600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa544388-f780-42d4-b295-6dacb0960ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:17.748330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa544388-f780-42d4-b295-6dacb0960ea8 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:17.748602 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa544388-f780-42d4-b295-6dacb0960ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:17.748878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa544388-f780-42d4-b295-6dacb0960ea8 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:17.749607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa544388-f780-42d4-b295-6dacb0960ea8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:17.749950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa544388-f780-42d4-b295-6dacb0960ea8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:17.751551 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa544388-f780-42d4-b295-6dacb0960ea8 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:17.771761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 380/749] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Oct 4 08:33:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:17.772930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa544388-f780-42d4-b295-6dacb0960ea8 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:17.788698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6665e7c-1f55-4a15-a9b9-6f6b8a9f94ce None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:17.828960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60686475-af85-4742-a3dd-4b29af0de722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:17.829587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60686475-af85-4742-a3dd-4b29af0de722 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:17.829911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60686475-af85-4742-a3dd-4b29af0de722 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:17.830181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60686475-af85-4742-a3dd-4b29af0de722 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:17.837378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60686475-af85-4742-a3dd-4b29af0de722 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:17.838861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60686475-af85-4742-a3dd-4b29af0de722 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-60686475-af85-4742-a3dd-4b29af0de722 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:17.851815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:17.852729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:17.853624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:17.853624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:17.853624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:17.853624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:17.853624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:17.853624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:17.853624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:17.853624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:17.853624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 370/750] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:17.854419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60686475-af85-4742-a3dd-4b29af0de722 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:17.858146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:17.894657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa544388-f780-42d4-b295-6dacb0960ea8 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:17.904036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:17.934063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73617dde-db0f-4b6f-b851-bff8a3274119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:17.934692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73617dde-db0f-4b6f-b851-bff8a3274119 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:17.934969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73617dde-db0f-4b6f-b851-bff8a3274119 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:17.935231 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73617dde-db0f-4b6f-b851-bff8a3274119 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:17.943203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73617dde-db0f-4b6f-b851-bff8a3274119 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:17.944690 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73617dde-db0f-4b6f-b851-bff8a3274119 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-73617dde-db0f-4b6f-b851-bff8a3274119 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:17.957188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:17.958418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:17.959407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:17.959407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:17.959407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:17.959407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:17.959407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:17.959407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:17.959407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:17.959407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:17.959407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:33:17.959407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:17.959407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 381/751] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 08:33:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:17.959976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb1900-7f17-4216-9f6f-3e7c990d107b None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:17.959976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb1900-7f17-4216-9f6f-3e7c990d107b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:17.960106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb1900-7f17-4216-9f6f-3e7c990d107b None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:17.961394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 382/752] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:17.961952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb1900-7f17-4216-9f6f-3e7c990d107b None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:17.972093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60686475-af85-4742-a3dd-4b29af0de722 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:18.009434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c027c-0191-4f74-b994-bef73fdb7a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:18.010065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c027c-0191-4f74-b994-bef73fdb7a19 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:18.010366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c027c-0191-4f74-b994-bef73fdb7a19 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:18.010618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c027c-0191-4f74-b994-bef73fdb7a19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:18.066839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb1900-7f17-4216-9f6f-3e7c990d107b None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:18.080875 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c027c-0191-4f74-b994-bef73fdb7a19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:18.082353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c027c-0191-4f74-b994-bef73fdb7a19 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:18.084418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:18.103034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26cc92-f79c-40ef-9910-6d7b12cd67bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:18.103636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26cc92-f79c-40ef-9910-6d7b12cd67bf None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:18.104181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26cc92-f79c-40ef-9910-6d7b12cd67bf None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:18.104458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26cc92-f79c-40ef-9910-6d7b12cd67bf None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:18.105146 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26cc92-f79c-40ef-9910-6d7b12cd67bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:18.105454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26cc92-f79c-40ef-9910-6d7b12cd67bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:18.106914 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26cc92-f79c-40ef-9910-6d7b12cd67bf None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:18.123439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 383/753] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:18.123439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26cc92-f79c-40ef-9910-6d7b12cd67bf None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:18.149338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:18.191293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 371/754] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:33:18.192556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83099b4f-8bb7-419d-9374-7d9606669672 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:18.192870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83099b4f-8bb7-419d-9374-7d9606669672 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:18.193749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83099b4f-8bb7-419d-9374-7d9606669672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:18.211376 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-83099b4f-8bb7-419d-9374-7d9606669672 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:18.212551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-83099b4f-8bb7-419d-9374-7d9606669672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:18.230503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26cc92-f79c-40ef-9910-6d7b12cd67bf None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:18.267232 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf31c05-e460-4350-9664-6ec2f2f7453c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:18.267808 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf31c05-e460-4350-9664-6ec2f2f7453c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:18.268293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf31c05-e460-4350-9664-6ec2f2f7453c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:18.268399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf31c05-e460-4350-9664-6ec2f2f7453c None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:18.269078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf31c05-e460-4350-9664-6ec2f2f7453c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:18.269386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf31c05-e460-4350-9664-6ec2f2f7453c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:18.270847 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf31c05-e460-4350-9664-6ec2f2f7453c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:18.283523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 372/755] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:18.284641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83099b4f-8bb7-419d-9374-7d9606669672 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:18.299136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 384/756] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Oct 4 08:33:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:18.299136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf31c05-e460-4350-9664-6ec2f2f7453c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:18.323524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:18.394829 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:18.402035 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83099b4f-8bb7-419d-9374-7d9606669672 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:18.412379 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf31c05-e460-4350-9664-6ec2f2f7453c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:18.437406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ca0d4-7c9c-4616-a655-e7ef1f97f290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:18.438199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ca0d4-7c9c-4616-a655-e7ef1f97f290 None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:18.438530 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ca0d4-7c9c-4616-a655-e7ef1f97f290 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:18.438872 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ca0d4-7c9c-4616-a655-e7ef1f97f290 None admin] PATH_INFO: `/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/ee8a19f4d9704408a8b737095e5885f1/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:18.441073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ca0d4-7c9c-4616-a655-e7ef1f97f290 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3df14a62714f6bab2acdfc979b655b, user_id=ee8a19f4d9704408a8b737095e5885f1, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:18.442706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ca0d4-7c9c-4616-a655-e7ef1f97f290 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:18.448934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae55ad-8dbc-4375-a323-4f4a0c1b84dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:18.449761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae55ad-8dbc-4375-a323-4f4a0c1b84dd None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:18.449950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae55ad-8dbc-4375-a323-4f4a0c1b84dd None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:18.450241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae55ad-8dbc-4375-a323-4f4a0c1b84dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:18.457583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 373/757] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:33:18 2022] PUT /identity/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/ee8a19f4d9704408a8b737095e5885f1/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:33:18.459262 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ca0d4-7c9c-4616-a655-e7ef1f97f290 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:18.461937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae55ad-8dbc-4375-a323-4f4a0c1b84dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:18.463460 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae55ad-8dbc-4375-a323-4f4a0c1b84dd None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-f8ae55ad-8dbc-4375-a323-4f4a0c1b84dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:18.491645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:18.492686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:18.493596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:18.493596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:18.493596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:18.493596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 385/758] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:18.498458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae55ad-8dbc-4375-a323-4f4a0c1b84dd None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:18.566941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:18.578237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ca0d4-7c9c-4616-a655-e7ef1f97f290 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:18.620195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298000fd-4ab5-4f88-883e-38a5fd5532d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:18.621168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298000fd-4ab5-4f88-883e-38a5fd5532d7 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:18.621637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298000fd-4ab5-4f88-883e-38a5fd5532d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:18.622106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298000fd-4ab5-4f88-883e-38a5fd5532d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:18.655917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae55ad-8dbc-4375-a323-4f4a0c1b84dd None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:18.699047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708c6e9-2133-4832-a553-3ad26c1a239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:18.699688 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c6e9-2133-4832-a553-3ad26c1a239d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:18.700046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c6e9-2133-4832-a553-3ad26c1a239d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:18.700388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c6e9-2133-4832-a553-3ad26c1a239d None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:18.701259 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708c6e9-2133-4832-a553-3ad26c1a239d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:18.701540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708c6e9-2133-4832-a553-3ad26c1a239d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:18.703608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298000fd-4ab5-4f88-883e-38a5fd5532d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:18.704045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708c6e9-2133-4832-a553-3ad26c1a239d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:18.705680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298000fd-4ab5-4f88-883e-38a5fd5532d7 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:18.706632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:18.717528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 386/759] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:18.718453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708c6e9-2133-4832-a553-3ad26c1a239d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:18.810102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:18.819103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 374/760] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:33:18.820012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298000fd-4ab5-4f88-883e-38a5fd5532d7 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:18.835857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708c6e9-2133-4832-a553-3ad26c1a239d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:18.880182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988a91b-f45f-488d-9795-eddb613eef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:18.881375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988a91b-f45f-488d-9795-eddb613eef37 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:18.881846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988a91b-f45f-488d-9795-eddb613eef37 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:18.882126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988a91b-f45f-488d-9795-eddb613eef37 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:18.883062 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988a91b-f45f-488d-9795-eddb613eef37 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:18.884877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988a91b-f45f-488d-9795-eddb613eef37 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-4988a91b-f45f-488d-9795-eddb613eef37 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:18.897204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:18.898279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:18.899322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 387/761] 10.222.0.26 () {56 vars in 1129 bytes} [Tue Oct 4 08:33:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:18.900123 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988a91b-f45f-488d-9795-eddb613eef37 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:18.937729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298000fd-4ab5-4f88-883e-38a5fd5532d7 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:18.978134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d7060-b1a2-4bb1-bf81-3bbe25e27f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:18.979126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d7060-b1a2-4bb1-bf81-3bbe25e27f52 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:18.979541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d7060-b1a2-4bb1-bf81-3bbe25e27f52 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:18.979910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d7060-b1a2-4bb1-bf81-3bbe25e27f52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:18.983182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:18.987881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d7060-b1a2-4bb1-bf81-3bbe25e27f52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:18.989660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d7060-b1a2-4bb1-bf81-3bbe25e27f52 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-1b0d7060-b1a2-4bb1-bf81-3bbe25e27f52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:19.003393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:19.004555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:19.006116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:19.006116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:19.006116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:19.006116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:19.006116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:19.006116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:19.006116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:19.006116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:19.006834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 375/762] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:19.007901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d7060-b1a2-4bb1-bf81-3bbe25e27f52 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:19.040875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988a91b-f45f-488d-9795-eddb613eef37 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:19.070632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:19.092600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0de6e9-292f-4229-b71c-107df81b69a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:19.093556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0de6e9-292f-4229-b71c-107df81b69a5 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:19.094068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0de6e9-292f-4229-b71c-107df81b69a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:19.094519 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0de6e9-292f-4229-b71c-107df81b69a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:19.109886 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0de6e9-292f-4229-b71c-107df81b69a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:19.112422 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0de6e9-292f-4229-b71c-107df81b69a5 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:19.123893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d7060-b1a2-4bb1-bf81-3bbe25e27f52 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-1c0de6e9-292f-4229-b71c-107df81b69a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:19.133033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:19.134028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:19.134915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:19.134915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:19.134915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:19.137482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 388/763] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:19.138765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0de6e9-292f-4229-b71c-107df81b69a5 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:19.163269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee0a96-2c27-472b-871e-18d169687737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:19.164120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee0a96-2c27-472b-871e-18d169687737 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:19.164495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee0a96-2c27-472b-871e-18d169687737 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:19.164902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee0a96-2c27-472b-871e-18d169687737 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:19.165711 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee0a96-2c27-472b-871e-18d169687737 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:19.166127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee0a96-2c27-472b-871e-18d169687737 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:19.167650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee0a96-2c27-472b-871e-18d169687737 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:19.177933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 376/764] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Oct 4 08:33:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:19.177933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee0a96-2c27-472b-871e-18d169687737 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:19.264230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0de6e9-292f-4229-b71c-107df81b69a5 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:19.278673 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:19.301706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cd235-af21-4e5a-b4c4-3c0fff382d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:19.302359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cd235-af21-4e5a-b4c4-3c0fff382d47 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:19.302640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cd235-af21-4e5a-b4c4-3c0fff382d47 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:19.302913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cd235-af21-4e5a-b4c4-3c0fff382d47 None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:19.303608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cd235-af21-4e5a-b4c4-3c0fff382d47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:19.303917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cd235-af21-4e5a-b4c4-3c0fff382d47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:19.305528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cd235-af21-4e5a-b4c4-3c0fff382d47 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:19.314121 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee0a96-2c27-472b-871e-18d169687737 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:19.322916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 389/765] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:19.322916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cd235-af21-4e5a-b4c4-3c0fff382d47 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:19.335122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:19.354881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7905e9-981d-4885-8106-4562a742d4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:19.355492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7905e9-981d-4885-8106-4562a742d4bf None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:19.355771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7905e9-981d-4885-8106-4562a742d4bf None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:19.356048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7905e9-981d-4885-8106-4562a742d4bf None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:19.356688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7905e9-981d-4885-8106-4562a742d4bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:19.356991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7905e9-981d-4885-8106-4562a742d4bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:19.358399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7905e9-981d-4885-8106-4562a742d4bf None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:19.375278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 377/766] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:19.375278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f103a92-2552-4ccd-964a-df803b7697f0 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:19.375278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f103a92-2552-4ccd-964a-df803b7697f0 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:19.375278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f103a92-2552-4ccd-964a-df803b7697f0 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:19.377879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 378/767] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:19.379081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f103a92-2552-4ccd-964a-df803b7697f0 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:19.426165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cd235-af21-4e5a-b4c4-3c0fff382d47 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:19.466291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cc286-8b92-4694-abb5-8a5d04bdc166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:19.467619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cc286-8b92-4694-abb5-8a5d04bdc166 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:19.468330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cc286-8b92-4694-abb5-8a5d04bdc166 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:19.468810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311cc286-8b92-4694-abb5-8a5d04bdc166 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:19.469790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cc286-8b92-4694-abb5-8a5d04bdc166 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:19.470569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cc286-8b92-4694-abb5-8a5d04bdc166 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:19.472118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311cc286-8b92-4694-abb5-8a5d04bdc166 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:19.480095 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 390/768] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Oct 4 08:33:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:19.481714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cc286-8b92-4694-abb5-8a5d04bdc166 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:19.486996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f103a92-2552-4ccd-964a-df803b7697f0 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:19.520476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:19.525161 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb1225-2ea1-4134-9efc-a1e75ed6d001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:19.526200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb1225-2ea1-4134-9efc-a1e75ed6d001 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:19.526483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb1225-2ea1-4134-9efc-a1e75ed6d001 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:19.526759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb1225-2ea1-4134-9efc-a1e75ed6d001 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:19.527478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb1225-2ea1-4134-9efc-a1e75ed6d001 None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=b232b71c77a24be5bfcdb6aeb5807f90) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:19.528142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb1225-2ea1-4134-9efc-a1e75ed6d001 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:19.529911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb1225-2ea1-4134-9efc-a1e75ed6d001 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:19.544157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 379/769] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:19 2022] GET /identity/v3/role_assignments?user.id=b232b71c77a24be5bfcdb6aeb5807f90&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:19.544157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657a839-f6eb-4cd1-9a70-6a998c72ead6 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:19.544157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657a839-f6eb-4cd1-9a70-6a998c72ead6 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:19.544157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657a839-f6eb-4cd1-9a70-6a998c72ead6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:19.558890 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c657a839-f6eb-4cd1-9a70-6a998c72ead6 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:19.560922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c657a839-f6eb-4cd1-9a70-6a998c72ead6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:19.602190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:19.605735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311cc286-8b92-4694-abb5-8a5d04bdc166 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:19.643403 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df41ca7-45ae-457d-83df-21b204cb510e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:19.644164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df41ca7-45ae-457d-83df-21b204cb510e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:19.644514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df41ca7-45ae-457d-83df-21b204cb510e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:19.644863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df41ca7-45ae-457d-83df-21b204cb510e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:19.646635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 380/770] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:19.647611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657a839-f6eb-4cd1-9a70-6a998c72ead6 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:19.653238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df41ca7-45ae-457d-83df-21b204cb510e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:19.654883 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df41ca7-45ae-457d-83df-21b204cb510e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-9df41ca7-45ae-457d-83df-21b204cb510e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:19.671441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:19.672795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:19.673669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:19.673669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:19.673669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:19.673669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:19.673669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:19.673669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:19.673669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:19.673669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:19.673669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 391/771] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:19.680285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df41ca7-45ae-457d-83df-21b204cb510e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:19.833155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:19.840098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df41ca7-45ae-457d-83df-21b204cb510e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:19.840408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657a839-f6eb-4cd1-9a70-6a998c72ead6 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:19.880099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2c00b-f455-4a06-a4a7-cd77ad04d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:19.881218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2c00b-f455-4a06-a4a7-cd77ad04d15d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:19.881713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2c00b-f455-4a06-a4a7-cd77ad04d15d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:19.882178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2c00b-f455-4a06-a4a7-cd77ad04d15d None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:19.883315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae2c00b-f455-4a06-a4a7-cd77ad04d15d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:19.883853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae2c00b-f455-4a06-a4a7-cd77ad04d15d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:19.885953 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae2c00b-f455-4a06-a4a7-cd77ad04d15d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:19.899756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 392/772] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:19.905761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2c00b-f455-4a06-a4a7-cd77ad04d15d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:19.931308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf90742-b782-4614-afcd-7570bfa54b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:19.932254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf90742-b782-4614-afcd-7570bfa54b9c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:19.932757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf90742-b782-4614-afcd-7570bfa54b9c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:19.933218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf90742-b782-4614-afcd-7570bfa54b9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:19.934586 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf90742-b782-4614-afcd-7570bfa54b9c None admin] RBAC: Adding query filter params (interface=public, service_id=c8c4748ca4cc45c7b6f7d5c869ec68a5, region_id=RegionOne) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:19.934871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf90742-b782-4614-afcd-7570bfa54b9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:19.936252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf90742-b782-4614-afcd-7570bfa54b9c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:19.960904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 381/773] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 08:33:19 2022] GET /identity/v3/endpoints?service_id=c8c4748ca4cc45c7b6f7d5c869ec68a5&interface=public®ion_id=RegionOne => generated 194 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:19.961986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf90742-b782-4614-afcd-7570bfa54b9c None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:20.001680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:20.029288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2c00b-f455-4a06-a4a7-cd77ad04d15d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:20.063407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6bb54-b1ab-4465-aeb5-c867cd1e7535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:20.064416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6bb54-b1ab-4465-aeb5-c867cd1e7535 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:20.064891 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6bb54-b1ab-4465-aeb5-c867cd1e7535 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:20.065370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6bb54-b1ab-4465-aeb5-c867cd1e7535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:20.066418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6bb54-b1ab-4465-aeb5-c867cd1e7535 None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=595dac02b846405cab34dcf898ce5b5a) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:20.067017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6bb54-b1ab-4465-aeb5-c867cd1e7535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:20.069182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6bb54-b1ab-4465-aeb5-c867cd1e7535 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:20.070048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf90742-b782-4614-afcd-7570bfa54b9c None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:20.075809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 393/774] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:19 2022] GET /identity/v3/role_assignments?user.id=595dac02b846405cab34dcf898ce5b5a&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:20.076930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6bb54-b1ab-4465-aeb5-c867cd1e7535 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:20.104817 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:20.118875 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5df96-75d3-4e82-9ec7-da27b92e135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:20.119556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5df96-75d3-4e82-9ec7-da27b92e135e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:20.119849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5df96-75d3-4e82-9ec7-da27b92e135e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:20.120235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5df96-75d3-4e82-9ec7-da27b92e135e None admin] PATH_INFO: `/v3/services/network` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:20.120846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5df96-75d3-4e82-9ec7-da27b92e135e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:20.122367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5df96-75d3-4e82-9ec7-da27b92e135e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-7cb5df96-75d3-4e82-9ec7-da27b92e135e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:20.135386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:20.136440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:20.137749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 382/775] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:19 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:20.142892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5df96-75d3-4e82-9ec7-da27b92e135e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:20.195020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6bb54-b1ab-4465-aeb5-c867cd1e7535 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:20.234472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6a306-e0c8-4e69-a59b-4ab40b1d5f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:20.235249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6a306-e0c8-4e69-a59b-4ab40b1d5f48 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:20.235528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6a306-e0c8-4e69-a59b-4ab40b1d5f48 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:20.235799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6a306-e0c8-4e69-a59b-4ab40b1d5f48 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:20.236417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6a306-e0c8-4e69-a59b-4ab40b1d5f48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:20.237834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6a306-e0c8-4e69-a59b-4ab40b1d5f48 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:20.245246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 394/776] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 08:33:20 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:20.302871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:20.308679 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5df96-75d3-4e82-9ec7-da27b92e135e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:20.346270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b1e1b-6248-4ec2-bf77-583797dbaef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:20.346902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b1e1b-6248-4ec2-bf77-583797dbaef8 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:20.347181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b1e1b-6248-4ec2-bf77-583797dbaef8 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:20.347450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b1e1b-6248-4ec2-bf77-583797dbaef8 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:20.348098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b1e1b-6248-4ec2-bf77-583797dbaef8 None admin] RBAC: Adding query filter params (name=network) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:20.348401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b1e1b-6248-4ec2-bf77-583797dbaef8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:20.350405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b1e1b-6248-4ec2-bf77-583797dbaef8 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:20.356581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 383/777] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:20.361903 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6a306-e0c8-4e69-a59b-4ab40b1d5f48 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:20.488700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:20.512151 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6a306-e0c8-4e69-a59b-4ab40b1d5f48 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:20.553531 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a296f67-e33d-44cf-8748-9a3efb42494d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:20.554187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a296f67-e33d-44cf-8748-9a3efb42494d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:20.554469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a296f67-e33d-44cf-8748-9a3efb42494d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:20.554822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a296f67-e33d-44cf-8748-9a3efb42494d None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:20.555485 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a296f67-e33d-44cf-8748-9a3efb42494d None admin] RBAC: Adding query filter params (type=network) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:20.555792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a296f67-e33d-44cf-8748-9a3efb42494d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:20.557225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a296f67-e33d-44cf-8748-9a3efb42494d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:20.565334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 395/778] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:20.571034 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b1e1b-6248-4ec2-bf77-583797dbaef8 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:20.680910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b1e1b-6248-4ec2-bf77-583797dbaef8 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:20.718066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab60a32-6833-400b-bcdf-26e9157694aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:20.718731 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab60a32-6833-400b-bcdf-26e9157694aa None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:20.719012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab60a32-6833-400b-bcdf-26e9157694aa None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:20.719285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab60a32-6833-400b-bcdf-26e9157694aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:20.720031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab60a32-6833-400b-bcdf-26e9157694aa None admin] RBAC: Adding query filter params (interface=public, service_id=accd82254cdc43c6ba3016f0aa49cb66, region_id=RegionOne) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:20.720343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab60a32-6833-400b-bcdf-26e9157694aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:20.721720 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab60a32-6833-400b-bcdf-26e9157694aa None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:20.727430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 384/779] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 08:33:20 2022] GET /identity/v3/endpoints?service_id=accd82254cdc43c6ba3016f0aa49cb66&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:20.734782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a296f67-e33d-44cf-8748-9a3efb42494d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:20.897968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a296f67-e33d-44cf-8748-9a3efb42494d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:20.948240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afbc67e-f2cc-480b-9cd9-5265658533af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:20.948791 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbc67e-f2cc-480b-9cd9-5265658533af None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:20.949066 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbc67e-f2cc-480b-9cd9-5265658533af None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:20.949332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbc67e-f2cc-480b-9cd9-5265658533af None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:20.949973 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbc67e-f2cc-480b-9cd9-5265658533af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:20.951353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbc67e-f2cc-480b-9cd9-5265658533af None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:20.957262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 396/780] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 08:33:20 2022] GET /identity/v3/services => generated 772 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:21.108951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0828e-b5dc-4936-bc9f-7cccc7c4288b None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:21.109521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0828e-b5dc-4936-bc9f-7cccc7c4288b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:21.109521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0828e-b5dc-4936-bc9f-7cccc7c4288b None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:21.111099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 385/781] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:21.117246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2c6d5-5959-4dcf-8fb5-6db23635eda4 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:21.117896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2c6d5-5959-4dcf-8fb5-6db23635eda4 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:21.118402 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2c6d5-5959-4dcf-8fb5-6db23635eda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:21.138812 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-88d2c6d5-5959-4dcf-8fb5-6db23635eda4 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:21.140529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-88d2c6d5-5959-4dcf-8fb5-6db23635eda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:21.150825 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:21.235085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 397/782] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:21.511458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0828e-b5dc-4936-bc9f-7cccc7c4288b None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:21.541932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:21.641177 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0828e-b5dc-4936-bc9f-7cccc7c4288b None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:21.708502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bcc03-bfee-4c4d-a414-52d3bc5467d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:21.709167 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bcc03-bfee-4c4d-a414-52d3bc5467d0 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:21.709449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bcc03-bfee-4c4d-a414-52d3bc5467d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:21.709741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bcc03-bfee-4c4d-a414-52d3bc5467d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:21.730579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0bcc03-bfee-4c4d-a414-52d3bc5467d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-8b0bcc03-bfee-4c4d-a414-52d3bc5467d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:21.735361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:21.736297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:21.737134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 386/783] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:33:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:21.744291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2c6d5-5959-4dcf-8fb5-6db23635eda4 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:21.908154 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2c6d5-5959-4dcf-8fb5-6db23635eda4 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:21.977374 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e59e38-af5d-4342-aef1-7428464a1f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:21.978387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e59e38-af5d-4342-aef1-7428464a1f51 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:21.979141 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e59e38-af5d-4342-aef1-7428464a1f51 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:21.979635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e59e38-af5d-4342-aef1-7428464a1f51 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:21.980492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e59e38-af5d-4342-aef1-7428464a1f51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:21.980997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e59e38-af5d-4342-aef1-7428464a1f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:21.982584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e59e38-af5d-4342-aef1-7428464a1f51 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:21.992520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 398/784] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:33:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:22.002800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bcc03-bfee-4c4d-a414-52d3bc5467d0 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:22.027887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d949879-01d8-4a05-9e25-582f3aa922df None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:22.028645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d949879-01d8-4a05-9e25-582f3aa922df None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:22.029120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d949879-01d8-4a05-9e25-582f3aa922df None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:22.030920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 399/785] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:22.035882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a78b9-5c01-4e60-8e5c-80295c59dd2d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:22.036163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a78b9-5c01-4e60-8e5c-80295c59dd2d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:22.036461 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a78b9-5c01-4e60-8e5c-80295c59dd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:22.050945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:22.072821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-912a78b9-5c01-4e60-8e5c-80295c59dd2d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:22.074039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-912a78b9-5c01-4e60-8e5c-80295c59dd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:22.162892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:22.180854 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bcc03-bfee-4c4d-a414-52d3bc5467d0 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:22.208320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 400/786] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:22.226393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ae107-772c-415b-b44b-a5cc56c9c499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:22.227032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ae107-772c-415b-b44b-a5cc56c9c499 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:22.227305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ae107-772c-415b-b44b-a5cc56c9c499 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:22.227571 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ae107-772c-415b-b44b-a5cc56c9c499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:22.245051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ae107-772c-415b-b44b-a5cc56c9c499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:22.246831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ae107-772c-415b-b44b-a5cc56c9c499 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-d56ae107-772c-415b-b44b-a5cc56c9c499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:22.269390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:22.270400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:22.271350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:22.271350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:22.271350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:22.271350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 387/787] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:22.277647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a78b9-5c01-4e60-8e5c-80295c59dd2d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:22.420546 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a78b9-5c01-4e60-8e5c-80295c59dd2d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:22.460527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f77da-1553-4fb5-824d-30497d037e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:22.461128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f77da-1553-4fb5-824d-30497d037e28 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:22.461411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f77da-1553-4fb5-824d-30497d037e28 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:22.461714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f77da-1553-4fb5-824d-30497d037e28 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:22.462358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f77da-1553-4fb5-824d-30497d037e28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:22.462663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f77da-1553-4fb5-824d-30497d037e28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:22.464107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f77da-1553-4fb5-824d-30497d037e28 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:22.465519 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:22.480764 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 401/788] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:22.486103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ae107-772c-415b-b44b-a5cc56c9c499 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:22.499260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f77da-1553-4fb5-824d-30497d037e28 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:22.679543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ae107-772c-415b-b44b-a5cc56c9c499 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:22.689179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f77da-1553-4fb5-824d-30497d037e28 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:22.733430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f45695-90b3-46b9-b59c-1237e8b83588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:22.734461 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f45695-90b3-46b9-b59c-1237e8b83588 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:22.734943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f45695-90b3-46b9-b59c-1237e8b83588 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:22.735414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f45695-90b3-46b9-b59c-1237e8b83588 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:22.737371 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b49b2b-ce14-4382-bfee-25a02c9c4aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:22.738039 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b49b2b-ce14-4382-bfee-25a02c9c4aab None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:22.738322 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b49b2b-ce14-4382-bfee-25a02c9c4aab None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:22.738599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b49b2b-ce14-4382-bfee-25a02c9c4aab None admin] PATH_INFO: `/v3/users/nova` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:22.747206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f45695-90b3-46b9-b59c-1237e8b83588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:22.748946 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b49b2b-ce14-4382-bfee-25a02c9c4aab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:22.750507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b49b2b-ce14-4382-bfee-25a02c9c4aab None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:22.753999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-46f45695-90b3-46b9-b59c-1237e8b83588 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:22.754599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:22.755550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:22.756393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:22.756393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:22.756393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:22.756393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:22.756393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:22.756393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:22.756393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:22.756393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:22.756393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:22.756393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:22.756393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:33:22.756393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:22.759115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 402/789] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:33:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:22.760607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe2289-0cf9-4c93-b57c-71263cb3b6bf None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:22.761091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe2289-0cf9-4c93-b57c-71263cb3b6bf None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:22.761589 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe2289-0cf9-4c93-b57c-71263cb3b6bf None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:22.763372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 403/790] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:22.771797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48187b13-c3e8-4996-891b-1033255e06c3 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-77b49b2b-ce14-4382-bfee-25a02c9c4aab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:22.771927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:22.773029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:22.774267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:22.774267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:22.774267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:22.774267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:22.774267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:22.774267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:22.774267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:22.774267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:22.774267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:33:22.774267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:22.774794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48187b13-c3e8-4996-891b-1033255e06c3 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:22.774794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48187b13-c3e8-4996-891b-1033255e06c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:22.774918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 388/791] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 08:33:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:22.774918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b49b2b-ce14-4382-bfee-25a02c9c4aab None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:22.794193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-48187b13-c3e8-4996-891b-1033255e06c3 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:22.796082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-48187b13-c3e8-4996-891b-1033255e06c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:22.853240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:22.877823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 404/792] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:22.878775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48187b13-c3e8-4996-891b-1033255e06c3 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:22.892232 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b49b2b-ce14-4382-bfee-25a02c9c4aab None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:22.935100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c3f90-c0a8-4481-b1d2-596be42148ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:22.936065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c3f90-c0a8-4481-b1d2-596be42148ac None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:22.936538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c3f90-c0a8-4481-b1d2-596be42148ac None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:22.936996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c3f90-c0a8-4481-b1d2-596be42148ac None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:22.937879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c3f90-c0a8-4481-b1d2-596be42148ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:22.938988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c3f90-c0a8-4481-b1d2-596be42148ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:22.940729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c3f90-c0a8-4481-b1d2-596be42148ac None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:22.963171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 389/793] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:33:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:22.968256 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c3f90-c0a8-4481-b1d2-596be42148ac None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:23.047310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48187b13-c3e8-4996-891b-1033255e06c3 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:23.094767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:23.114015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2de67-238d-4139-bac7-61071e0a4d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:23.115211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2de67-238d-4139-bac7-61071e0a4d29 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.115498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2de67-238d-4139-bac7-61071e0a4d29 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.115772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2de67-238d-4139-bac7-61071e0a4d29 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.116518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2de67-238d-4139-bac7-61071e0a4d29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:23.116830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2de67-238d-4139-bac7-61071e0a4d29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:23.118327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2de67-238d-4139-bac7-61071e0a4d29 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:23.155213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 405/794] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 08:33:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:23.161216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2de67-238d-4139-bac7-61071e0a4d29 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:23.174709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c3f90-c0a8-4481-b1d2-596be42148ac None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:23.223179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb5c47-62e5-4e06-af62-89bf61d06349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:23.223729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb5c47-62e5-4e06-af62-89bf61d06349 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.224545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb5c47-62e5-4e06-af62-89bf61d06349 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.224821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb5c47-62e5-4e06-af62-89bf61d06349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.231355 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:23.236617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb5c47-62e5-4e06-af62-89bf61d06349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:23.238172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb5c47-62e5-4e06-af62-89bf61d06349 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-79cb5c47-62e5-4e06-af62-89bf61d06349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:23.254910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:23.255952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:23.256826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:23.256826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:23.256826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:23.260633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 390/795] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:23.261990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117135ea-39db-4e5d-a36c-5a83a7901b1e None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.262275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117135ea-39db-4e5d-a36c-5a83a7901b1e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.262552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117135ea-39db-4e5d-a36c-5a83a7901b1e None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.263858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 391/796] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:23.264629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117135ea-39db-4e5d-a36c-5a83a7901b1e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:23.296583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2de67-238d-4139-bac7-61071e0a4d29 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:23.334644 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05d91c-5180-4de0-a0ec-9dba405cfe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:23.335266 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05d91c-5180-4de0-a0ec-9dba405cfe9a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.335544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05d91c-5180-4de0-a0ec-9dba405cfe9a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.335823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05d91c-5180-4de0-a0ec-9dba405cfe9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.376502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117135ea-39db-4e5d-a36c-5a83a7901b1e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:23.390845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:23.409812 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05d91c-5180-4de0-a0ec-9dba405cfe9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:23.411500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05d91c-5180-4de0-a0ec-9dba405cfe9a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:23.415627 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7b7e1-2468-452f-8f87-88886e56eb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:23.416264 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7b7e1-2468-452f-8f87-88886e56eb83 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.416544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7b7e1-2468-452f-8f87-88886e56eb83 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.416815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7b7e1-2468-452f-8f87-88886e56eb83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.429393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7b7e1-2468-452f-8f87-88886e56eb83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-87b7b7e1-2468-452f-8f87-88886e56eb83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:23.435105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:23.436213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:23.437247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:23.437247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:23.437247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:23.437247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:23.437247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:23.437247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:23.437247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:23.437247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:23.437247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:23.437247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:23.437247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:23.437247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:23.439331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 392/797] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:33:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:23.444154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7b7e1-2468-452f-8f87-88886e56eb83 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:23.455618 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:23.518809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 406/798] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:33:23.520133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea337f-7076-4baf-bc6a-3ef7b33984cc None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.520430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea337f-7076-4baf-bc6a-3ef7b33984cc None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.520708 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea337f-7076-4baf-bc6a-3ef7b33984cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.538484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-9dea337f-7076-4baf-bc6a-3ef7b33984cc None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:23.539677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9dea337f-7076-4baf-bc6a-3ef7b33984cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:23.554714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7b7e1-2468-452f-8f87-88886e56eb83 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:23.589757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d0c0d-4efb-47fa-9a20-8bd95f89987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:23.590360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d0c0d-4efb-47fa-9a20-8bd95f89987e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.590641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d0c0d-4efb-47fa-9a20-8bd95f89987e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.590912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d0c0d-4efb-47fa-9a20-8bd95f89987e None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.591547 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d0c0d-4efb-47fa-9a20-8bd95f89987e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:23.591849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d0c0d-4efb-47fa-9a20-8bd95f89987e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:23.593206 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d0c0d-4efb-47fa-9a20-8bd95f89987e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:23.609949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 393/799] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:23.609949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d0c0d-4efb-47fa-9a20-8bd95f89987e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:23.611935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 407/800] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:23.612777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea337f-7076-4baf-bc6a-3ef7b33984cc None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:23.659492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:23.695768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:23.735212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea337f-7076-4baf-bc6a-3ef7b33984cc None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:23.745555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d0c0d-4efb-47fa-9a20-8bd95f89987e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:23.770443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600b6b8-4b19-4bdf-9fa8-697221735b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:23.771582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600b6b8-4b19-4bdf-9fa8-697221735b82 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.771858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600b6b8-4b19-4bdf-9fa8-697221735b82 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.772128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600b6b8-4b19-4bdf-9fa8-697221735b82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.782209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600b6b8-4b19-4bdf-9fa8-697221735b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:23.783646 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600b6b8-4b19-4bdf-9fa8-697221735b82 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:23.787777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25a2a6-4db7-4139-9356-8456fdf29688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:23.788651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25a2a6-4db7-4139-9356-8456fdf29688 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.789123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25a2a6-4db7-4139-9356-8456fdf29688 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.789667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25a2a6-4db7-4139-9356-8456fdf29688 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.790555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25a2a6-4db7-4139-9356-8456fdf29688 None admin] RBAC: Adding query filter params (name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:23.791547 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25a2a6-4db7-4139-9356-8456fdf29688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:23.793785 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25a2a6-4db7-4139-9356-8456fdf29688 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-c600b6b8-4b19-4bdf-9fa8-697221735b82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:23.798854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:23.799979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:23.800892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:23.800892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:23.800892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:23.800892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 408/801] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:23.800892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600b6b8-4b19-4bdf-9fa8-697221735b82 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:23.809315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 394/802] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:33:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:23.810762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f702c01-8922-42cc-adb0-2c4ef5fda1fc None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.811181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f702c01-8922-42cc-adb0-2c4ef5fda1fc None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.811586 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f702c01-8922-42cc-adb0-2c4ef5fda1fc None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.812898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 395/803] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:23.813739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f702c01-8922-42cc-adb0-2c4ef5fda1fc None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:23.912394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600b6b8-4b19-4bdf-9fa8-697221735b82 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:23.917322 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:23.924990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f702c01-8922-42cc-adb0-2c4ef5fda1fc None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:23.934832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:23.947120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a195220-2bb0-40e6-9fbf-ba202a41d11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:23.947739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a195220-2bb0-40e6-9fbf-ba202a41d11a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.948014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a195220-2bb0-40e6-9fbf-ba202a41d11a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.948282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a195220-2bb0-40e6-9fbf-ba202a41d11a None admin] PATH_INFO: `/v3/users/swift` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.955524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a195220-2bb0-40e6-9fbf-ba202a41d11a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:23.956960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a195220-2bb0-40e6-9fbf-ba202a41d11a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:23.958122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f85ae-b9c1-431f-9f2d-f4013753dbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:23.959297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f85ae-b9c1-431f-9f2d-f4013753dbf6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.959575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f85ae-b9c1-431f-9f2d-f4013753dbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.959840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f85ae-b9c1-431f-9f2d-f4013753dbf6 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.960474 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f85ae-b9c1-431f-9f2d-f4013753dbf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:23.960773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f85ae-b9c1-431f-9f2d-f4013753dbf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:23.962173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f85ae-b9c1-431f-9f2d-f4013753dbf6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-8a195220-2bb0-40e6-9fbf-ba202a41d11a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:23.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:23.969483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:23.970452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:23.970452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:23.970452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:23.970452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:23.970452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:23.970452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:23.970452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:23.970452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:23.970452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:33:23.970452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:23.970452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 409/804] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:33:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:23.970452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a195220-2bb0-40e6-9fbf-ba202a41d11a None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:23.977194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 396/805] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:23.978558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df0196-6c81-4df6-8bf1-9b4c03e0ef44 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:23.979126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df0196-6c81-4df6-8bf1-9b4c03e0ef44 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:23.979395 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df0196-6c81-4df6-8bf1-9b4c03e0ef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:23.996728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-21df0196-6c81-4df6-8bf1-9b4c03e0ef44 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:23.997970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-21df0196-6c81-4df6-8bf1-9b4c03e0ef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:24.062219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 397/806] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:24.062962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df0196-6c81-4df6-8bf1-9b4c03e0ef44 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:24.074469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a195220-2bb0-40e6-9fbf-ba202a41d11a None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:24.109438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d6cb7-36a4-4790-a04a-400ac3567007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:24.110089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d6cb7-36a4-4790-a04a-400ac3567007 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:24.110368 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d6cb7-36a4-4790-a04a-400ac3567007 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:24.110639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d6cb7-36a4-4790-a04a-400ac3567007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:24.120926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d6cb7-36a4-4790-a04a-400ac3567007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:24.122396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d6cb7-36a4-4790-a04a-400ac3567007 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-1d9d6cb7-36a4-4790-a04a-400ac3567007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:24.137008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:24.137965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:24.139171 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:24.139171 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:24.139171 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:24.139171 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 410/807] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:24.139171 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d6cb7-36a4-4790-a04a-400ac3567007 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:24.165611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:24.178443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:24.183076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df0196-6c81-4df6-8bf1-9b4c03e0ef44 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:24.218242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c6eef-55d1-4f80-b9bd-1a49ba078f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:24.218867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c6eef-55d1-4f80-b9bd-1a49ba078f8e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:24.219146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c6eef-55d1-4f80-b9bd-1a49ba078f8e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:24.219414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c6eef-55d1-4f80-b9bd-1a49ba078f8e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:24.220054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c6eef-55d1-4f80-b9bd-1a49ba078f8e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:24.221483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c6eef-55d1-4f80-b9bd-1a49ba078f8e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-dd3c6eef-55d1-4f80-b9bd-1a49ba078f8e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:24.231314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:24.232289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:24.233742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 398/808] 10.222.0.26 () {56 vars in 1129 bytes} [Tue Oct 4 08:33:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:24.234555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c6eef-55d1-4f80-b9bd-1a49ba078f8e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:24.250277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d6cb7-36a4-4790-a04a-400ac3567007 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:24.287056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b100a-c013-4efc-a006-48ecd0af2b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:24.287664 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b100a-c013-4efc-a006-48ecd0af2b55 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:24.287943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b100a-c013-4efc-a006-48ecd0af2b55 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:24.288214 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b100a-c013-4efc-a006-48ecd0af2b55 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:24.288933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b100a-c013-4efc-a006-48ecd0af2b55 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:24.289238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b100a-c013-4efc-a006-48ecd0af2b55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:24.290687 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b100a-c013-4efc-a006-48ecd0af2b55 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:24.312830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 411/809] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Oct 4 08:33:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:24.312830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b100a-c013-4efc-a006-48ecd0af2b55 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:24.342658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c6eef-55d1-4f80-b9bd-1a49ba078f8e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:24.376480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f61cda-8b35-4076-8a5f-75742e70b15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:24.377302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61cda-8b35-4076-8a5f-75742e70b15a None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:24.377584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61cda-8b35-4076-8a5f-75742e70b15a None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:24.377898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61cda-8b35-4076-8a5f-75742e70b15a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:24.384869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f61cda-8b35-4076-8a5f-75742e70b15a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:24.386341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f61cda-8b35-4076-8a5f-75742e70b15a None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-a1f61cda-8b35-4076-8a5f-75742e70b15a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:24.398007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:24.399157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:24.400065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:24.400065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:24.400065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:24.400065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:24.400065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:24.400065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:24.400065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:24.400065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:24.400065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 399/810] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:24.400065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f61cda-8b35-4076-8a5f-75742e70b15a None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:24.407177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:24.410570 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:24.414844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b100a-c013-4efc-a006-48ecd0af2b55 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:24.449598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de46a3-8612-4763-bac2-f28148effc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:24.450229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de46a3-8612-4763-bac2-f28148effc3b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:24.450510 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de46a3-8612-4763-bac2-f28148effc3b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:24.450783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de46a3-8612-4763-bac2-f28148effc3b None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:24.451420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de46a3-8612-4763-bac2-f28148effc3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:24.451726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de46a3-8612-4763-bac2-f28148effc3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:24.453095 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de46a3-8612-4763-bac2-f28148effc3b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:24.464248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 412/811] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:24.464917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de46a3-8612-4763-bac2-f28148effc3b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:24.502289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f61cda-8b35-4076-8a5f-75742e70b15a None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:24.535325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc580843-f059-4af4-9a21-9643c0e759f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:24.535921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc580843-f059-4af4-9a21-9643c0e759f5 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:24.536198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc580843-f059-4af4-9a21-9643c0e759f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:24.537033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc580843-f059-4af4-9a21-9643c0e759f5 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:24.537654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc580843-f059-4af4-9a21-9643c0e759f5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:24.537990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc580843-f059-4af4-9a21-9643c0e759f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:24.539343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc580843-f059-4af4-9a21-9643c0e759f5 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:24.543680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 400/812] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Oct 4 08:33:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:24.544447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc580843-f059-4af4-9a21-9643c0e759f5 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:24.560397 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de46a3-8612-4763-bac2-f28148effc3b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:24.593780 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6be15-96d6-41e9-ae81-b61c6e8880de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:24.594347 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6be15-96d6-41e9-ae81-b61c6e8880de None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:24.594628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6be15-96d6-41e9-ae81-b61c6e8880de None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:24.594897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6be15-96d6-41e9-ae81-b61c6e8880de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:24.605444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6be15-96d6-41e9-ae81-b61c6e8880de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:24.606925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6be15-96d6-41e9-ae81-b61c6e8880de None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:24.613882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-37a6be15-96d6-41e9-ae81-b61c6e8880de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:24.620809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:24.621972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:24.623061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:24.623061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:24.623061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:24.625998 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 413/813] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:24.626824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6be15-96d6-41e9-ae81-b61c6e8880de None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:24.636061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:24.644176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc580843-f059-4af4-9a21-9643c0e759f5 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:24.679058 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f82c5d-6d5a-4ff4-b954-a0f66aef61c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:24.679684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f82c5d-6d5a-4ff4-b954-a0f66aef61c6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:24.679960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f82c5d-6d5a-4ff4-b954-a0f66aef61c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:24.680229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f82c5d-6d5a-4ff4-b954-a0f66aef61c6 None admin] PATH_INFO: `/v3/services/network` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:24.680801 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f82c5d-6d5a-4ff4-b954-a0f66aef61c6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:24.682363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f82c5d-6d5a-4ff4-b954-a0f66aef61c6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-44f82c5d-6d5a-4ff4-b954-a0f66aef61c6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:24.690882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:24.692008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:24.693532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 401/814] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:24 2022] GET /identity/v3/services/network => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:24.694573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f82c5d-6d5a-4ff4-b954-a0f66aef61c6 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:24.740184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6be15-96d6-41e9-ae81-b61c6e8880de None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:24.773173 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83680acd-4eff-471b-ae7b-de91786f5b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:24.773799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83680acd-4eff-471b-ae7b-de91786f5b85 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:24.774070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83680acd-4eff-471b-ae7b-de91786f5b85 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:24.774343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83680acd-4eff-471b-ae7b-de91786f5b85 None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:24.775022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83680acd-4eff-471b-ae7b-de91786f5b85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:24.775329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83680acd-4eff-471b-ae7b-de91786f5b85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:24.776764 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83680acd-4eff-471b-ae7b-de91786f5b85 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:24.788880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 414/815] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:24.789670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83680acd-4eff-471b-ae7b-de91786f5b85 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:24.792023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f82c5d-6d5a-4ff4-b954-a0f66aef61c6 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:24.826154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7569a1-b6c5-482b-9133-a509ef193f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:24.826779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7569a1-b6c5-482b-9133-a509ef193f06 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:24.827055 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7569a1-b6c5-482b-9133-a509ef193f06 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:24.827326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7569a1-b6c5-482b-9133-a509ef193f06 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:24.829961 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:24.835178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7569a1-b6c5-482b-9133-a509ef193f06 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:24.836816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7569a1-b6c5-482b-9133-a509ef193f06 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-ce7569a1-b6c5-482b-9133-a509ef193f06 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:24.847826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:24.849436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:24.850506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:24.850506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:24.850506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:24.850506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:24.850506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:24.850506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:24.850506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:24.850506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:24.850506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:33:24.850506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:24.853248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:24.854403 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 402/816] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 08:33:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:24.855359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7569a1-b6c5-482b-9133-a509ef193f06 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:24.887733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83680acd-4eff-471b-ae7b-de91786f5b85 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:24.921061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b28bd-11e2-439b-80d3-8369406f78aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:24.921665 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b28bd-11e2-439b-80d3-8369406f78aa None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:24.921973 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b28bd-11e2-439b-80d3-8369406f78aa None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:24.922247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b28bd-11e2-439b-80d3-8369406f78aa None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:24.922870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b28bd-11e2-439b-80d3-8369406f78aa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:24.923173 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b28bd-11e2-439b-80d3-8369406f78aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:24.924531 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b28bd-11e2-439b-80d3-8369406f78aa None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:24.929659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 415/817] 10.222.0.26 () {56 vars in 1124 bytes} [Tue Oct 4 08:33:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:24.930475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b28bd-11e2-439b-80d3-8369406f78aa None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:24.957287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7569a1-b6c5-482b-9133-a509ef193f06 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:24.991597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59e6c3-63c5-4488-8339-34c0ad8f3721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:24.992200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59e6c3-63c5-4488-8339-34c0ad8f3721 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:24.992489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59e6c3-63c5-4488-8339-34c0ad8f3721 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:24.992762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59e6c3-63c5-4488-8339-34c0ad8f3721 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:24.993396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59e6c3-63c5-4488-8339-34c0ad8f3721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:24.993723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59e6c3-63c5-4488-8339-34c0ad8f3721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:24.995083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59e6c3-63c5-4488-8339-34c0ad8f3721 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:25.006448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 403/818] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:25.007234 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59e6c3-63c5-4488-8339-34c0ad8f3721 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:25.028457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b28bd-11e2-439b-80d3-8369406f78aa None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:25.056435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:25.062988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6210853-5d60-42f9-90cd-841270b758fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:25.063583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6210853-5d60-42f9-90cd-841270b758fd None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:25.063860 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6210853-5d60-42f9-90cd-841270b758fd None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:25.064126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6210853-5d60-42f9-90cd-841270b758fd None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:25.064455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:25.065114 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6210853-5d60-42f9-90cd-841270b758fd None admin] RBAC: Adding query filter params (name=network) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:25.065114 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6210853-5d60-42f9-90cd-841270b758fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:25.066627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6210853-5d60-42f9-90cd-841270b758fd None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:25.071147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 416/819] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:25.071918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6210853-5d60-42f9-90cd-841270b758fd None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:25.109515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59e6c3-63c5-4488-8339-34c0ad8f3721 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:25.142543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637012a-b839-414a-97da-c92032e2dd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:25.143139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637012a-b839-414a-97da-c92032e2dd0e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:25.143416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637012a-b839-414a-97da-c92032e2dd0e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:25.143689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637012a-b839-414a-97da-c92032e2dd0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:25.144394 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637012a-b839-414a-97da-c92032e2dd0e None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=ee8a19f4d9704408a8b737095e5885f1) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:25.144704 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637012a-b839-414a-97da-c92032e2dd0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:25.146674 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637012a-b839-414a-97da-c92032e2dd0e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:25.152199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 404/820] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:25 2022] GET /identity/v3/role_assignments?user.id=ee8a19f4d9704408a8b737095e5885f1&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:25.152950 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637012a-b839-414a-97da-c92032e2dd0e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:25.180717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6210853-5d60-42f9-90cd-841270b758fd None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:25.218924 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4e663-c24a-44b7-a299-3a20a146badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:25.219423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e663-c24a-44b7-a299-3a20a146badc None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:25.219707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e663-c24a-44b7-a299-3a20a146badc None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:25.219980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e663-c24a-44b7-a299-3a20a146badc None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:25.220720 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4e663-c24a-44b7-a299-3a20a146badc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:25.221033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4e663-c24a-44b7-a299-3a20a146badc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:25.222526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4e663-c24a-44b7-a299-3a20a146badc None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:25.242552 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 417/821] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Oct 4 08:33:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:25.243449 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4e663-c24a-44b7-a299-3a20a146badc None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:25.276726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637012a-b839-414a-97da-c92032e2dd0e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:25.299566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:25.300244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:25.319151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7a410-2be2-4020-9512-a48eb5f20e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:25.319765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7a410-2be2-4020-9512-a48eb5f20e82 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:25.320052 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7a410-2be2-4020-9512-a48eb5f20e82 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:25.320317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7a410-2be2-4020-9512-a48eb5f20e82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:25.320992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7a410-2be2-4020-9512-a48eb5f20e82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:25.322404 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7a410-2be2-4020-9512-a48eb5f20e82 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:25.347678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 405/822] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:33:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:33:25.348514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7a410-2be2-4020-9512-a48eb5f20e82 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:25.358561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4e663-c24a-44b7-a299-3a20a146badc None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:25.416625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f89bb-2268-45a3-b35c-b0e99a0c176a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:25.417192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f89bb-2268-45a3-b35c-b0e99a0c176a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:25.417478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f89bb-2268-45a3-b35c-b0e99a0c176a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:25.417783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f89bb-2268-45a3-b35c-b0e99a0c176a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:25.425560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f89bb-2268-45a3-b35c-b0e99a0c176a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:25.427260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f89bb-2268-45a3-b35c-b0e99a0c176a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-ae9f89bb-2268-45a3-b35c-b0e99a0c176a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:25.439925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:25.440979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:25.442458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:25.442458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:25.442458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:25.442458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:25.442458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:25.442458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:25.442458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:25.442458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:25.442458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 418/823] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:25.446248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f89bb-2268-45a3-b35c-b0e99a0c176a None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:25.571308 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f89bb-2268-45a3-b35c-b0e99a0c176a None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:25.583230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:25.612381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca1770-b115-4144-80e1-df56bae3be7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:25.613075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca1770-b115-4144-80e1-df56bae3be7b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:25.613361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca1770-b115-4144-80e1-df56bae3be7b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:25.613638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca1770-b115-4144-80e1-df56bae3be7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:25.618281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7a410-2be2-4020-9512-a48eb5f20e82 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:25.625349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca1770-b115-4144-80e1-df56bae3be7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:25.627025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca1770-b115-4144-80e1-df56bae3be7b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-13ca1770-b115-4144-80e1-df56bae3be7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:25.643393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:25.644585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:25.645453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:25.645453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:25.645453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:25.645453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 419/824] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:25.645453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca1770-b115-4144-80e1-df56bae3be7b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:25.671006 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598a28f-6f6d-44e6-9bc4-ec899ca5f393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:25.671554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598a28f-6f6d-44e6-9bc4-ec899ca5f393 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:25.671832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598a28f-6f6d-44e6-9bc4-ec899ca5f393 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:25.672103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598a28f-6f6d-44e6-9bc4-ec899ca5f393 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:25.672736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598a28f-6f6d-44e6-9bc4-ec899ca5f393 None admin] RBAC: Adding query filter params (type=network) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:25.673092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598a28f-6f6d-44e6-9bc4-ec899ca5f393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:25.674487 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598a28f-6f6d-44e6-9bc4-ec899ca5f393 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:25.680462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 406/825] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:25.681270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598a28f-6f6d-44e6-9bc4-ec899ca5f393 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:25.726473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:25.790502 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca1770-b115-4144-80e1-df56bae3be7b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:25.791623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598a28f-6f6d-44e6-9bc4-ec899ca5f393 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:25.828122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9b91e-9673-43e7-9d1b-75736665a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:25.828739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9b91e-9673-43e7-9d1b-75736665a210 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:25.829015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9b91e-9673-43e7-9d1b-75736665a210 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:25.829328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9b91e-9673-43e7-9d1b-75736665a210 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:25.830012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9b91e-9673-43e7-9d1b-75736665a210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:25.830313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9b91e-9673-43e7-9d1b-75736665a210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:25.831870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9b91e-9673-43e7-9d1b-75736665a210 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:25.837843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007a668-11bf-4d54-809f-bd892973c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:25.838761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007a668-11bf-4d54-809f-bd892973c892 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:25.839208 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007a668-11bf-4d54-809f-bd892973c892 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:25.840175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007a668-11bf-4d54-809f-bd892973c892 None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:25.841018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007a668-11bf-4d54-809f-bd892973c892 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:25.842008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007a668-11bf-4d54-809f-bd892973c892 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:25.843941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007a668-11bf-4d54-809f-bd892973c892 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:25.844816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 407/826] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:25.845669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9b91e-9673-43e7-9d1b-75736665a210 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:25.859597 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 420/827] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:25.860258 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007a668-11bf-4d54-809f-bd892973c892 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:25.884668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:25.970546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:25.973326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007a668-11bf-4d54-809f-bd892973c892 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:25.974266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9b91e-9673-43e7-9d1b-75736665a210 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:26.010008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df22b1f-4dbc-4001-8731-8abf68fd6ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:26.010891 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df22b1f-4dbc-4001-8731-8abf68fd6ee3 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:26.011320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df22b1f-4dbc-4001-8731-8abf68fd6ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:26.011769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df22b1f-4dbc-4001-8731-8abf68fd6ee3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:26.017602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb6720-1178-4396-8d85-0b144dab2192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:26.018237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb6720-1178-4396-8d85-0b144dab2192 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:26.018517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb6720-1178-4396-8d85-0b144dab2192 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:26.018790 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb6720-1178-4396-8d85-0b144dab2192 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:26.019754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb6720-1178-4396-8d85-0b144dab2192 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:26.020380 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df22b1f-4dbc-4001-8731-8abf68fd6ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:26.021660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb6720-1178-4396-8d85-0b144dab2192 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:26.022694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df22b1f-4dbc-4001-8731-8abf68fd6ee3 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-9df22b1f-4dbc-4001-8731-8abf68fd6ee3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:26.034680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:26.035857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:26.037063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:26.037063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:26.037063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:26.037063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:26.037063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:26.037063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:26.037063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:26.037063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:26.037063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 421/828] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:26.037063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df22b1f-4dbc-4001-8731-8abf68fd6ee3 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:26.052386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 408/829] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:33:25 2022] POST /identity/v3/endpoints => generated 337 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:33:26.053819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb6720-1178-4396-8d85-0b144dab2192 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:26.225559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:26.256087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb6720-1178-4396-8d85-0b144dab2192 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:26.270987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df22b1f-4dbc-4001-8731-8abf68fd6ee3 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:26.305849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad402b3d-3ec6-42d7-b242-0d7268457deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:26.306624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad402b3d-3ec6-42d7-b242-0d7268457deb None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:26.306995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad402b3d-3ec6-42d7-b242-0d7268457deb None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:26.307359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad402b3d-3ec6-42d7-b242-0d7268457deb None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:26.308308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad402b3d-3ec6-42d7-b242-0d7268457deb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:26.309008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad402b3d-3ec6-42d7-b242-0d7268457deb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:26.310832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad402b3d-3ec6-42d7-b242-0d7268457deb None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:26.313452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dd697-24b8-4710-b418-c66a4e6cec87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:26.314152 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dd697-24b8-4710-b418-c66a4e6cec87 None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:26.314431 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dd697-24b8-4710-b418-c66a4e6cec87 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:26.314708 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dd697-24b8-4710-b418-c66a4e6cec87 None admin] PATH_INFO: `/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/b232b71c77a24be5bfcdb6aeb5807f90/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:26.316607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679dd697-24b8-4710-b418-c66a4e6cec87 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3df14a62714f6bab2acdfc979b655b, user_id=b232b71c77a24be5bfcdb6aeb5807f90, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:26.318635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679dd697-24b8-4710-b418-c66a4e6cec87 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:26.326532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 409/830] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:26.332726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad402b3d-3ec6-42d7-b242-0d7268457deb None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:26.338922 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 422/831] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:33:26 2022] PUT /identity/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/b232b71c77a24be5bfcdb6aeb5807f90/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:33:26.349933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:26.457541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad402b3d-3ec6-42d7-b242-0d7268457deb None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:26.497388 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed9002-4258-4c79-8621-a33215c4c1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:26.498003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed9002-4258-4c79-8621-a33215c4c1d5 None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:26.498423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed9002-4258-4c79-8621-a33215c4c1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:26.498703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed9002-4258-4c79-8621-a33215c4c1d5 None admin] PATH_INFO: `/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/595dac02b846405cab34dcf898ce5b5a/roles/a01bc9944e4f46a4a9a2b16f510541c9` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:26.515618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed9002-4258-4c79-8621-a33215c4c1d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3df14a62714f6bab2acdfc979b655b, user_id=595dac02b846405cab34dcf898ce5b5a, role_id=a01bc9944e4f46a4a9a2b16f510541c9)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:26.517074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed9002-4258-4c79-8621-a33215c4c1d5 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:26.535214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 410/832] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:33:26 2022] PUT /identity/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/595dac02b846405cab34dcf898ce5b5a/roles/a01bc9944e4f46a4a9a2b16f510541c9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:33:27.495413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85daca26-5a46-4d9f-b036-35c4bc019b3e None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:27.495949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85daca26-5a46-4d9f-b036-35c4bc019b3e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:27.495949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85daca26-5a46-4d9f-b036-35c4bc019b3e None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:27.497432 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 423/833] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:27.502819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8878ef-cd7e-4799-a4e7-a394b865784f None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:27.503208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8878ef-cd7e-4799-a4e7-a394b865784f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:27.503372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8878ef-cd7e-4799-a4e7-a394b865784f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:27.521593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-6f8878ef-cd7e-4799-a4e7-a394b865784f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:27.522857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6f8878ef-cd7e-4799-a4e7-a394b865784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:27.583879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:27.595239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 411/834] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:27.674144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083ab35-d7e2-448a-baa6-2290171cf75a None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:27.674429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083ab35-d7e2-448a-baa6-2290171cf75a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:27.674702 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083ab35-d7e2-448a-baa6-2290171cf75a None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:27.676290 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 424/835] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:27.681521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19520be-145e-43fa-8124-3b40fc54f23c None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:27.681829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19520be-145e-43fa-8124-3b40fc54f23c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:27.682101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19520be-145e-43fa-8124-3b40fc54f23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:27.701779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-e19520be-145e-43fa-8124-3b40fc54f23c None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:27.703310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e19520be-145e-43fa-8124-3b40fc54f23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:27.774214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 412/836] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:27.878881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083ab35-d7e2-448a-baa6-2290171cf75a None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:27.986953 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083ab35-d7e2-448a-baa6-2290171cf75a None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:28.030011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb22c9-d5cf-4386-89b8-536a45d7fc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:28.030987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb22c9-d5cf-4386-89b8-536a45d7fc32 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:28.031469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb22c9-d5cf-4386-89b8-536a45d7fc32 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:28.031952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb22c9-d5cf-4386-89b8-536a45d7fc32 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:28.032732 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb22c9-d5cf-4386-89b8-536a45d7fc32 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:28.034348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb22c9-d5cf-4386-89b8-536a45d7fc32 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-9cdb22c9-d5cf-4386-89b8-536a45d7fc32 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:28.043500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:28.044412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:33:28.045301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:28.046077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19520be-145e-43fa-8124-3b40fc54f23c None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:28.048051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 425/837] 10.222.0.26 () {56 vars in 1132 bytes} [Tue Oct 4 08:33:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:28.057747 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb22c9-d5cf-4386-89b8-536a45d7fc32 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:28.057887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:28.153975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19520be-145e-43fa-8124-3b40fc54f23c None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:28.178575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:28.191948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b2831-9d12-4614-891c-97cba5fc4692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:28.192735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b2831-9d12-4614-891c-97cba5fc4692 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:28.193084 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b2831-9d12-4614-891c-97cba5fc4692 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:28.193426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b2831-9d12-4614-891c-97cba5fc4692 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:28.194115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b2831-9d12-4614-891c-97cba5fc4692 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:28.195557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b2831-9d12-4614-891c-97cba5fc4692 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:28.198960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb22c9-d5cf-4386-89b8-536a45d7fc32 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-307b2831-9d12-4614-891c-97cba5fc4692 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:28.205171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:28.206496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:33:28.207647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:28.209185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 413/838] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:33:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:28.215291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b2831-9d12-4614-891c-97cba5fc4692 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:28.252185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c733155-114e-486a-afed-3d0d767fd2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:28.252795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c733155-114e-486a-afed-3d0d767fd2a5 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:28.253069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c733155-114e-486a-afed-3d0d767fd2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:28.253336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c733155-114e-486a-afed-3d0d767fd2a5 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:28.253993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c733155-114e-486a-afed-3d0d767fd2a5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:28.254297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c733155-114e-486a-afed-3d0d767fd2a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:28.255828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c733155-114e-486a-afed-3d0d767fd2a5 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:28.261721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 426/839] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 08:33:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:28.266362 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c733155-114e-486a-afed-3d0d767fd2a5 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:28.337366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:28.350187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b2831-9d12-4614-891c-97cba5fc4692 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:28.373931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c733155-114e-486a-afed-3d0d767fd2a5 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:28.390874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8423d-c7eb-4b49-814a-634553d43586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:28.391492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8423d-c7eb-4b49-814a-634553d43586 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:28.391773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8423d-c7eb-4b49-814a-634553d43586 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:28.392042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8423d-c7eb-4b49-814a-634553d43586 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:28.392773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8423d-c7eb-4b49-814a-634553d43586 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:28.393145 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8423d-c7eb-4b49-814a-634553d43586 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:28.394573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8423d-c7eb-4b49-814a-634553d43586 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:28.404614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 414/840] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:33:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:28.406328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8423d-c7eb-4b49-814a-634553d43586 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:28.416577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847b7e2-0a5e-4f07-93f4-7a7f8a9f170f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:28.417809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847b7e2-0a5e-4f07-93f4-7a7f8a9f170f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:28.418099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847b7e2-0a5e-4f07-93f4-7a7f8a9f170f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:28.418377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847b7e2-0a5e-4f07-93f4-7a7f8a9f170f None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:28.419018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847b7e2-0a5e-4f07-93f4-7a7f8a9f170f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:28.419325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847b7e2-0a5e-4f07-93f4-7a7f8a9f170f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:28.421197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847b7e2-0a5e-4f07-93f4-7a7f8a9f170f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:28.429769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 427/841] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 08:33:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:28.474006 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:28.533594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8423d-c7eb-4b49-814a-634553d43586 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:28.578994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4fb286-1f8b-4c28-9521-87bfbb6f43a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:28.579983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fb286-1f8b-4c28-9521-87bfbb6f43a1 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:28.580495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fb286-1f8b-4c28-9521-87bfbb6f43a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:28.580973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4fb286-1f8b-4c28-9521-87bfbb6f43a1 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:28.581858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fb286-1f8b-4c28-9521-87bfbb6f43a1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:28.582392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fb286-1f8b-4c28-9521-87bfbb6f43a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:28.584050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4fb286-1f8b-4c28-9521-87bfbb6f43a1 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:28.590114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 415/842] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:33:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:28.646231 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596fc58-0411-4f5b-abb8-698ffb856ff3 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:28.646766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596fc58-0411-4f5b-abb8-698ffb856ff3 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:28.646766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596fc58-0411-4f5b-abb8-698ffb856ff3 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:28.649179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 428/843] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:28.656589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96051816-e2f7-4d65-9730-59d4975d4e5d None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:28.656883 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96051816-e2f7-4d65-9730-59d4975d4e5d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:28.657161 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96051816-e2f7-4d65-9730-59d4975d4e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:28.678922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-96051816-e2f7-4d65-9730-59d4975d4e5d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:28.680193 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-96051816-e2f7-4d65-9730-59d4975d4e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:28.757915 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 416/844] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:28.812557 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5726c21-07c6-4f3d-afc5-35a54f322a92 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:28.812877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5726c21-07c6-4f3d-afc5-35a54f322a92 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:28.813120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5726c21-07c6-4f3d-afc5-35a54f322a92 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:28.814781 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 429/845] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:28.820173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d12fb0-cf8f-4699-980e-767bd8d2c1ff None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:28.820465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d12fb0-cf8f-4699-980e-767bd8d2c1ff None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:28.820741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d12fb0-cf8f-4699-980e-767bd8d2c1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:28.839965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-88d12fb0-cf8f-4699-980e-767bd8d2c1ff None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:28.841560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-88d12fb0-cf8f-4699-980e-767bd8d2c1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:28.885137 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:28.914767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 417/846] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:29.025637 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5726c21-07c6-4f3d-afc5-35a54f322a92 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:29.134725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5726c21-07c6-4f3d-afc5-35a54f322a92 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:29.166775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:29.173211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbc6d7-07ea-422d-ac31-894e2602c58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:29.173876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbc6d7-07ea-422d-ac31-894e2602c58a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:29.174160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbc6d7-07ea-422d-ac31-894e2602c58a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:29.174485 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbc6d7-07ea-422d-ac31-894e2602c58a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:29.186069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbc6d7-07ea-422d-ac31-894e2602c58a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-1bfbc6d7-07ea-422d-ac31-894e2602c58a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:29.191389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:29.193721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:29.195407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 430/847] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:33:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:29.199712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d12fb0-cf8f-4699-980e-767bd8d2c1ff None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:29.246619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbc6d7-07ea-422d-ac31-894e2602c58a None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:29.314490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d12fb0-cf8f-4699-980e-767bd8d2c1ff None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:29.354499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbc6d7-07ea-422d-ac31-894e2602c58a None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:29.355826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e29b1-19fa-4f84-8ab3-a4dd31bfbf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:29.356439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e29b1-19fa-4f84-8ab3-a4dd31bfbf3b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:29.356733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e29b1-19fa-4f84-8ab3-a4dd31bfbf3b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:29.357161 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e29b1-19fa-4f84-8ab3-a4dd31bfbf3b None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:29.357854 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e29b1-19fa-4f84-8ab3-a4dd31bfbf3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:29.358969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e29b1-19fa-4f84-8ab3-a4dd31bfbf3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:29.360362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e29b1-19fa-4f84-8ab3-a4dd31bfbf3b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:29.373729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 418/848] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:33:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:29.377060 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e29b1-19fa-4f84-8ab3-a4dd31bfbf3b None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:29.403466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcd62c-8054-4760-afcf-1eaf93b8ef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:29.404418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcd62c-8054-4760-afcf-1eaf93b8ef3f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:29.404882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcd62c-8054-4760-afcf-1eaf93b8ef3f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:29.405338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcd62c-8054-4760-afcf-1eaf93b8ef3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:29.421448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:29.424797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcd62c-8054-4760-afcf-1eaf93b8ef3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-cadcd62c-8054-4760-afcf-1eaf93b8ef3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:29.433537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:29.434582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:29.435424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:29.435424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:29.435424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:29.435424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:29.435424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:29.435424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:29.435424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:29.435424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:29.435424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:29.435424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:29.435424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:33:29.435424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:29.443831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 431/849] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:33:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:29.444581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcd62c-8054-4760-afcf-1eaf93b8ef3f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:29.504514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e29b1-19fa-4f84-8ab3-a4dd31bfbf3b None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:29.523840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:29.551597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9c834-7569-4b86-a6b9-4263a658e619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:29.553551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9c834-7569-4b86-a6b9-4263a658e619 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:29.554054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9c834-7569-4b86-a6b9-4263a658e619 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:29.554521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9c834-7569-4b86-a6b9-4263a658e619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:29.571102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9c834-7569-4b86-a6b9-4263a658e619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:29.573140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9c834-7569-4b86-a6b9-4263a658e619 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:29.583862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcd62c-8054-4760-afcf-1eaf93b8ef3f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-2cc9c834-7569-4b86-a6b9-4263a658e619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:29.592792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:29.593749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:29.594637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:29.594637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:29.594637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:29.597134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 419/850] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:29.602495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9c834-7569-4b86-a6b9-4263a658e619 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:29.626392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734175c-bb01-4378-ab78-c561e94a142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:29.627083 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734175c-bb01-4378-ab78-c561e94a142c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:29.627573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734175c-bb01-4378-ab78-c561e94a142c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:29.627869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734175c-bb01-4378-ab78-c561e94a142c None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:29.628626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734175c-bb01-4378-ab78-c561e94a142c None admin] RBAC: Adding query filter params (name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:29.629058 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734175c-bb01-4378-ab78-c561e94a142c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:29.630585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734175c-bb01-4378-ab78-c561e94a142c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:29.644501 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 432/851] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:33:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:29.645254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734175c-bb01-4378-ab78-c561e94a142c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:29.716283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:29.730315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9c834-7569-4b86-a6b9-4263a658e619 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:29.766811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44e5f9-ab99-43c7-87b0-29ef00ae70b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:29.767424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44e5f9-ab99-43c7-87b0-29ef00ae70b2 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:29.767703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44e5f9-ab99-43c7-87b0-29ef00ae70b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:29.767974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44e5f9-ab99-43c7-87b0-29ef00ae70b2 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:29.768611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44e5f9-ab99-43c7-87b0-29ef00ae70b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:29.768969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44e5f9-ab99-43c7-87b0-29ef00ae70b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:29.770413 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44e5f9-ab99-43c7-87b0-29ef00ae70b2 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:29.785024 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734175c-bb01-4378-ab78-c561e94a142c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:29.787741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 420/852] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:29.788673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44e5f9-ab99-43c7-87b0-29ef00ae70b2 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:29.804940 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:29.831442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046d7a5-4da0-4d9c-b45c-be1795078275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:29.832452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046d7a5-4da0-4d9c-b45c-be1795078275 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:29.833054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046d7a5-4da0-4d9c-b45c-be1795078275 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:29.834572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046d7a5-4da0-4d9c-b45c-be1795078275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:29.848426 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046d7a5-4da0-4d9c-b45c-be1795078275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:29.850759 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f046d7a5-4da0-4d9c-b45c-be1795078275 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-f046d7a5-4da0-4d9c-b45c-be1795078275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:29.869226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:29.870221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:29.871108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:29.871108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:29.871108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:29.872969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 433/853] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:29.878495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046d7a5-4da0-4d9c-b45c-be1795078275 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:29.900515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44e5f9-ab99-43c7-87b0-29ef00ae70b2 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:29.946005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7d7c3-5c78-4530-a1c6-030822cab04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:29.946987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7d7c3-5c78-4530-a1c6-030822cab04e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:29.947462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7d7c3-5c78-4530-a1c6-030822cab04e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:29.947941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7d7c3-5c78-4530-a1c6-030822cab04e None admin] PATH_INFO: `/v3/users/swift` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:29.957320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7d7c3-5c78-4530-a1c6-030822cab04e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:29.958826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7d7c3-5c78-4530-a1c6-030822cab04e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-06a7d7c3-5c78-4530-a1c6-030822cab04e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:29.973176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:29.974693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:29.975609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:29.975609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:29.975609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:29.975609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:29.975609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:29.975609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:29.975609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:29.975609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:29.975609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 08:33:29.975609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:29.975609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 421/854] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:33:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:29.981257 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7d7c3-5c78-4530-a1c6-030822cab04e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:30.001659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f046d7a5-4da0-4d9c-b45c-be1795078275 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:30.018043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:30.045725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8cec2-65b0-4ecf-9d02-67e6b01eb7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:30.046633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8cec2-65b0-4ecf-9d02-67e6b01eb7f3 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:30.047006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8cec2-65b0-4ecf-9d02-67e6b01eb7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:30.047374 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8cec2-65b0-4ecf-9d02-67e6b01eb7f3 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:30.048650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8cec2-65b0-4ecf-9d02-67e6b01eb7f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:30.049060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8cec2-65b0-4ecf-9d02-67e6b01eb7f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:30.050591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8cec2-65b0-4ecf-9d02-67e6b01eb7f3 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:30.070427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 434/855] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:30.072404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8cec2-65b0-4ecf-9d02-67e6b01eb7f3 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:30.095404 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7d7c3-5c78-4530-a1c6-030822cab04e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:30.106665 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:30.140090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57555bf7-b5d6-4d42-b5e9-a7e9726bccba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:30.141203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57555bf7-b5d6-4d42-b5e9-a7e9726bccba None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:30.141813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57555bf7-b5d6-4d42-b5e9-a7e9726bccba None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:30.142379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57555bf7-b5d6-4d42-b5e9-a7e9726bccba None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:30.143322 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57555bf7-b5d6-4d42-b5e9-a7e9726bccba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:30.143845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57555bf7-b5d6-4d42-b5e9-a7e9726bccba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:30.147021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57555bf7-b5d6-4d42-b5e9-a7e9726bccba None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:30.171793 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 422/856] 10.222.0.26 () {56 vars in 1135 bytes} [Tue Oct 4 08:33:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:30.173408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57555bf7-b5d6-4d42-b5e9-a7e9726bccba None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:30.194548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8cec2-65b0-4ecf-9d02-67e6b01eb7f3 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:30.236937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a339-fff7-4548-8219-49a2575a1db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:30.237496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a339-fff7-4548-8219-49a2575a1db4 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:30.237802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a339-fff7-4548-8219-49a2575a1db4 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:30.238071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a339-fff7-4548-8219-49a2575a1db4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:30.248179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8a339-fff7-4548-8219-49a2575a1db4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:30.249911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8a339-fff7-4548-8219-49a2575a1db4 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:30.251812 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-c2f8a339-fff7-4548-8219-49a2575a1db4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:30.264547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:30.265520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:30.266490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:30.266490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:30.266490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:30.266490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:30.266490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:30.266490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:30.266490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:30.266490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:30.266490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:33:30.266490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:30.269821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 435/857] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 08:33:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:30.273459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a339-fff7-4548-8219-49a2575a1db4 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:30.308011 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57555bf7-b5d6-4d42-b5e9-a7e9726bccba None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:30.350629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f01004-0017-4654-8ad7-bf98db28a068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:30.351590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f01004-0017-4654-8ad7-bf98db28a068 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:30.352065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f01004-0017-4654-8ad7-bf98db28a068 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:30.352531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f01004-0017-4654-8ad7-bf98db28a068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:30.364090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:30.381359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a339-fff7-4548-8219-49a2575a1db4 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:30.416467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc9564-09cf-4e4e-b109-954a8ba07175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:30.417104 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc9564-09cf-4e4e-b109-954a8ba07175 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:30.417383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc9564-09cf-4e4e-b109-954a8ba07175 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:30.417658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc9564-09cf-4e4e-b109-954a8ba07175 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:30.418531 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc9564-09cf-4e4e-b109-954a8ba07175 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:30.418723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc9564-09cf-4e4e-b109-954a8ba07175 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:30.420178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc9564-09cf-4e4e-b109-954a8ba07175 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:30.437670 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f01004-0017-4654-8ad7-bf98db28a068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:30.443580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 436/858] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Oct 4 08:33:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:30.443711 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f01004-0017-4654-8ad7-bf98db28a068 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:30.444906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc9564-09cf-4e4e-b109-954a8ba07175 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:30.520416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:30.561546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 423/859] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:30 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:33:30.562574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f01004-0017-4654-8ad7-bf98db28a068 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:30.577265 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc9564-09cf-4e4e-b109-954a8ba07175 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:30.630358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecb7a8-54f5-427c-8501-5203f5430f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:30.631427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecb7a8-54f5-427c-8501-5203f5430f4c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:30.632186 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecb7a8-54f5-427c-8501-5203f5430f4c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:30.633036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecb7a8-54f5-427c-8501-5203f5430f4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:30.652455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:30.683919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f01004-0017-4654-8ad7-bf98db28a068 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:30.720001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecb7a8-54f5-427c-8501-5203f5430f4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:30.721454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecb7a8-54f5-427c-8501-5203f5430f4c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:30.722341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73dd6e2-56d5-450f-9225-d80fbaba4049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:30.723018 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dd6e2-56d5-450f-9225-d80fbaba4049 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:30.723456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dd6e2-56d5-450f-9225-d80fbaba4049 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:30.723759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dd6e2-56d5-450f-9225-d80fbaba4049 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:30.736519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73dd6e2-56d5-450f-9225-d80fbaba4049 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:30.738018 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73dd6e2-56d5-450f-9225-d80fbaba4049 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-f73dd6e2-56d5-450f-9225-d80fbaba4049 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:30.757198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:30.758244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:30.759147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:30.759147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:30.759147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:30.759147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 424/860] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:30.759966 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2e160-feff-4a9f-8607-163155bcd27c None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:30.759966 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2e160-feff-4a9f-8607-163155bcd27c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:30.760204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2e160-feff-4a9f-8607-163155bcd27c None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:30.761744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 425/861] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:30.764182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2e160-feff-4a9f-8607-163155bcd27c None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:30.787776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:30.835601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 437/862] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:30 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:33:30.837401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0effeb5-a532-401b-bc53-78d86cd2d031 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:30.837863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0effeb5-a532-401b-bc53-78d86cd2d031 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:30.838347 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0effeb5-a532-401b-bc53-78d86cd2d031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:30.858584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d0effeb5-a532-401b-bc53-78d86cd2d031 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:30.859785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d0effeb5-a532-401b-bc53-78d86cd2d031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:30.878254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2e160-feff-4a9f-8607-163155bcd27c None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:30.900818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:30.912930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c435977-a568-4134-9c38-5640c11f482f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:30.913528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c435977-a568-4134-9c38-5640c11f482f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:30.913836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c435977-a568-4134-9c38-5640c11f482f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:30.914105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c435977-a568-4134-9c38-5640c11f482f None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:30.914730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c435977-a568-4134-9c38-5640c11f482f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:30.915028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c435977-a568-4134-9c38-5640c11f482f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:30.916374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c435977-a568-4134-9c38-5640c11f482f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:30.928020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 438/863] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:30.928179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 426/864] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:30.928731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0effeb5-a532-401b-bc53-78d86cd2d031 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:30.930920 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044610b4-d4b2-4831-ab49-0340f82d78e2 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:30.931203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044610b4-d4b2-4831-ab49-0340f82d78e2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:30.931474 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044610b4-d4b2-4831-ab49-0340f82d78e2 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:30.932792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 427/865] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:30.933478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044610b4-d4b2-4831-ab49-0340f82d78e2 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:31.023325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:31.040936 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0effeb5-a532-401b-bc53-78d86cd2d031 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:31.046574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044610b4-d4b2-4831-ab49-0340f82d78e2 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:31.076182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ffd1b-8105-428d-966c-d986238b1f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:31.076808 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ffd1b-8105-428d-966c-d986238b1f1f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:31.077089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ffd1b-8105-428d-966c-d986238b1f1f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:31.077363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ffd1b-8105-428d-966c-d986238b1f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:31.082416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3faa20-03fe-4769-a3d5-964ea1e43a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:31.083051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3faa20-03fe-4769-a3d5-964ea1e43a68 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:31.083327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3faa20-03fe-4769-a3d5-964ea1e43a68 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:31.083593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3faa20-03fe-4769-a3d5-964ea1e43a68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:31.087539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ffd1b-8105-428d-966c-d986238b1f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:31.088977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ffd1b-8105-428d-966c-d986238b1f1f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:31.090300 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3faa20-03fe-4769-a3d5-964ea1e43a68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:31.091704 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3faa20-03fe-4769-a3d5-964ea1e43a68 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-4a3faa20-03fe-4769-a3d5-964ea1e43a68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:31.102701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:31.103576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:31.104732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:31.104732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:31.104732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:31.104732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:31.104732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:31.104732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:31.104732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:31.104732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-e58ffd1b-8105-428d-966c-d986238b1f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:31.105177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:31.106047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:31.106910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:31.106910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:31.106910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:31.108835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 428/866] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:31.109464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 439/867] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:31.109906 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d5aae-20ff-4280-827a-782bdd9371a9 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:31.110186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d5aae-20ff-4280-827a-782bdd9371a9 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:31.110334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ffd1b-8105-428d-966c-d986238b1f1f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:31.110502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d5aae-20ff-4280-827a-782bdd9371a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:31.129362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-ae2d5aae-20ff-4280-827a-782bdd9371a9 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:31.131157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ae2d5aae-20ff-4280-827a-782bdd9371a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:31.139732 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:31.203747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 429/868] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:31.205362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d5aae-20ff-4280-827a-782bdd9371a9 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:31.221087 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ffd1b-8105-428d-966c-d986238b1f1f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:31.258018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5310d-4948-42b4-ad71-7448af0dd984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:31.258621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5310d-4948-42b4-ad71-7448af0dd984 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:31.258908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5310d-4948-42b4-ad71-7448af0dd984 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:31.259163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c5310d-4948-42b4-ad71-7448af0dd984 None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:31.259853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5310d-4948-42b4-ad71-7448af0dd984 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:31.260156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5310d-4948-42b4-ad71-7448af0dd984 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:31.261580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c5310d-4948-42b4-ad71-7448af0dd984 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:31.279174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 440/869] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:31.280005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5310d-4948-42b4-ad71-7448af0dd984 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:31.281945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:31.320370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d5aae-20ff-4280-827a-782bdd9371a9 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:31.356663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7a294-4a37-476c-9a79-84532eba5886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:31.357268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7a294-4a37-476c-9a79-84532eba5886 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:31.357548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7a294-4a37-476c-9a79-84532eba5886 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:31.357851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7a294-4a37-476c-9a79-84532eba5886 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:31.358482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7a294-4a37-476c-9a79-84532eba5886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:31.358780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7a294-4a37-476c-9a79-84532eba5886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:31.360122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7a294-4a37-476c-9a79-84532eba5886 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:31.368144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:31.377420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 430/870] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:31.377420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7a294-4a37-476c-9a79-84532eba5886 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:31.389168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c5310d-4948-42b4-ad71-7448af0dd984 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:31.425309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1777a70-2708-40cd-a0b5-5ea3d5607ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:31.425936 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1777a70-2708-40cd-a0b5-5ea3d5607ae3 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:31.426216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1777a70-2708-40cd-a0b5-5ea3d5607ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:31.426484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1777a70-2708-40cd-a0b5-5ea3d5607ae3 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:31.427331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1777a70-2708-40cd-a0b5-5ea3d5607ae3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:31.428688 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1777a70-2708-40cd-a0b5-5ea3d5607ae3 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:31.445301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 441/871] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:33:31 2022] POST /identity/v3/services => generated 269 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:33:31.446181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1777a70-2708-40cd-a0b5-5ea3d5607ae3 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:31.492138 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7a294-4a37-476c-9a79-84532eba5886 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:31.524190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:31.543583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85069401-4718-49e9-b8d0-35115d1e8c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:31.544308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85069401-4718-49e9-b8d0-35115d1e8c55 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:31.544594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85069401-4718-49e9-b8d0-35115d1e8c55 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:31.544859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85069401-4718-49e9-b8d0-35115d1e8c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:31.545601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85069401-4718-49e9-b8d0-35115d1e8c55 None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=b232b71c77a24be5bfcdb6aeb5807f90) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:31.545954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85069401-4718-49e9-b8d0-35115d1e8c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:31.547400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85069401-4718-49e9-b8d0-35115d1e8c55 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:31.554558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 431/872] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:31 2022] GET /identity/v3/role_assignments?user.id=b232b71c77a24be5bfcdb6aeb5807f90&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:31.555399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85069401-4718-49e9-b8d0-35115d1e8c55 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:31.584923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1777a70-2708-40cd-a0b5-5ea3d5607ae3 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:31.633053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937b1c7-fab6-4eec-9027-ed607a7133ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:31.633782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937b1c7-fab6-4eec-9027-ed607a7133ff None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:31.634064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937b1c7-fab6-4eec-9027-ed607a7133ff None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:31.634345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937b1c7-fab6-4eec-9027-ed607a7133ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:31.644150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937b1c7-fab6-4eec-9027-ed607a7133ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:31.646263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937b1c7-fab6-4eec-9027-ed607a7133ff None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-5937b1c7-fab6-4eec-9027-ed607a7133ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:31.662168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:31.663129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:31.664008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:31.664008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:31.664008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:31.664008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:31.664008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:31.664008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:31.664008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:31.664008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:31.664008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 442/873] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:31.670139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937b1c7-fab6-4eec-9027-ed607a7133ff None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:31.681299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:31.697499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85069401-4718-49e9-b8d0-35115d1e8c55 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:31.743420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e06af-a210-469d-851a-f3dc1fd03542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:31.745024 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e06af-a210-469d-851a-f3dc1fd03542 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:31.745487 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e06af-a210-469d-851a-f3dc1fd03542 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:31.746293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e06af-a210-469d-851a-f3dc1fd03542 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:31.747212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e06af-a210-469d-851a-f3dc1fd03542 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:31.748774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e06af-a210-469d-851a-f3dc1fd03542 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:31.767536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 432/874] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:33:31 2022] POST /identity/v3/services => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:33:31.798715 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937b1c7-fab6-4eec-9027-ed607a7133ff None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:31.848781 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a6ef7-af8e-487a-9620-dbf15f5da222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:31.850198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a6ef7-af8e-487a-9620-dbf15f5da222 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:31.850661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a6ef7-af8e-487a-9620-dbf15f5da222 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:31.851107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a6ef7-af8e-487a-9620-dbf15f5da222 None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:31.852074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a6ef7-af8e-487a-9620-dbf15f5da222 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:31.852574 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a6ef7-af8e-487a-9620-dbf15f5da222 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:31.854545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a6ef7-af8e-487a-9620-dbf15f5da222 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:31.871056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 443/875] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:31.877199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e06af-a210-469d-851a-f3dc1fd03542 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:31.937831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:32.005796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e06af-a210-469d-851a-f3dc1fd03542 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:32.042684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b5c52-c651-422f-bd42-77c809e4e6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:32.043696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b5c52-c651-422f-bd42-77c809e4e6b8 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:32.044176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b5c52-c651-422f-bd42-77c809e4e6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:32.044643 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b5c52-c651-422f-bd42-77c809e4e6b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:32.045585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b5c52-c651-422f-bd42-77c809e4e6b8 None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=595dac02b846405cab34dcf898ce5b5a) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:32.046186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b5c52-c651-422f-bd42-77c809e4e6b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:32.047862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b5c52-c651-422f-bd42-77c809e4e6b8 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:32.054669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 433/876] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:31 2022] GET /identity/v3/role_assignments?user.id=595dac02b846405cab34dcf898ce5b5a&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:33.724722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ff496-07a6-4ea5-97c7-4523751f58b3 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:33.725677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ff496-07a6-4ea5-97c7-4523751f58b3 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:33.726190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ff496-07a6-4ea5-97c7-4523751f58b3 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:33.727956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 444/877] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:33.733655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5279ff1-73ab-40be-b878-5a1a659190c1 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:33.734545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5279ff1-73ab-40be-b878-5a1a659190c1 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:33.735031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5279ff1-73ab-40be-b878-5a1a659190c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:33.754527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-b5279ff1-73ab-40be-b878-5a1a659190c1 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:33.756124 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b5279ff1-73ab-40be-b878-5a1a659190c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:33.812793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0a880-b932-44c4-a50b-0e553e97e4c1 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:33.813120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0a880-b932-44c4-a50b-0e553e97e4c1 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:33.813356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0a880-b932-44c4-a50b-0e553e97e4c1 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:33.814993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 445/878] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:33:33.820218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15aa08d-db5e-4300-b1ea-e9bc14dafe84 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:33.820508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15aa08d-db5e-4300-b1ea-e9bc14dafe84 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:33.820786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15aa08d-db5e-4300-b1ea-e9bc14dafe84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:33.826693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 434/879] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:33.839286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-f15aa08d-db5e-4300-b1ea-e9bc14dafe84 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:33.840491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f15aa08d-db5e-4300-b1ea-e9bc14dafe84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:33.900712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:33.917983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 446/880] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:34.034640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efb235-addf-4557-97d8-3de000ed3a9e None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:34.034957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efb235-addf-4557-97d8-3de000ed3a9e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:34.035198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2efb235-addf-4557-97d8-3de000ed3a9e None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:34.037447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 435/881] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:34.042601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519041da-43d0-413e-ab8e-ce9e67028523 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:34.042895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519041da-43d0-413e-ab8e-ce9e67028523 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:34.043176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519041da-43d0-413e-ab8e-ce9e67028523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:34.061580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-519041da-43d0-413e-ab8e-ce9e67028523 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:34.062802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-519041da-43d0-413e-ab8e-ce9e67028523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:34.114208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efb235-addf-4557-97d8-3de000ed3a9e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:34.137292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 447/882] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:34.160271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:34.192943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519041da-43d0-413e-ab8e-ce9e67028523 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:34.231551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2efb235-addf-4557-97d8-3de000ed3a9e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:34.268917 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c7ae2-7128-474a-b197-e8b2073ab348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:34.269549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c7ae2-7128-474a-b197-e8b2073ab348 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:34.269862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c7ae2-7128-474a-b197-e8b2073ab348 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:34.270135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c7ae2-7128-474a-b197-e8b2073ab348 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:34.271396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c7ae2-7128-474a-b197-e8b2073ab348 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:34.273265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c7ae2-7128-474a-b197-e8b2073ab348 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-cc1c7ae2-7128-474a-b197-e8b2073ab348 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:34.282884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:34.283819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:34.285096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 436/883] 10.222.0.26 () {56 vars in 1132 bytes} [Tue Oct 4 08:33:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:34.291986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c7ae2-7128-474a-b197-e8b2073ab348 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:34.328564 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519041da-43d0-413e-ab8e-ce9e67028523 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:34.352713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:34.366211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c947765-b4c0-4e77-87d3-a627986074cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:34.366925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c947765-b4c0-4e77-87d3-a627986074cf None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:34.367271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c947765-b4c0-4e77-87d3-a627986074cf None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:34.367601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c947765-b4c0-4e77-87d3-a627986074cf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:34.368249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c947765-b4c0-4e77-87d3-a627986074cf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:34.369608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c947765-b4c0-4e77-87d3-a627986074cf None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-9c947765-b4c0-4e77-87d3-a627986074cf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:34.378297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:34.379274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:34.380641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 448/884] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 08:33:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:34.381416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc534b64-59f6-4395-ac61-0c4add80374f None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:34.381416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc534b64-59f6-4395-ac61-0c4add80374f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:34.381554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc534b64-59f6-4395-ac61-0c4add80374f None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:34.383599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 449/885] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:34.387406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc534b64-59f6-4395-ac61-0c4add80374f None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:34.409566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c7ae2-7128-474a-b197-e8b2073ab348 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:34.417259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:34.445073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa73a-f43d-406e-b64e-0e7ee805b8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:34.445933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa73a-f43d-406e-b64e-0e7ee805b8c0 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:34.446215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa73a-f43d-406e-b64e-0e7ee805b8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:34.446489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa73a-f43d-406e-b64e-0e7ee805b8c0 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:34.447110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa73a-f43d-406e-b64e-0e7ee805b8c0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:34.447417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa73a-f43d-406e-b64e-0e7ee805b8c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:34.448772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa73a-f43d-406e-b64e-0e7ee805b8c0 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:34.453441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 437/886] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 08:33:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:34.454588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764cfad-0e82-47fc-9b38-fc9c1507f05a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:34.454874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764cfad-0e82-47fc-9b38-fc9c1507f05a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:34.455144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764cfad-0e82-47fc-9b38-fc9c1507f05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:34.473796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-8764cfad-0e82-47fc-9b38-fc9c1507f05a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:34.474978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8764cfad-0e82-47fc-9b38-fc9c1507f05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:34.491374 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc534b64-59f6-4395-ac61-0c4add80374f None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:34.524657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b26645-0a9f-4be3-b7d6-0ec1805bc06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:34.525245 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b26645-0a9f-4be3-b7d6-0ec1805bc06d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:34.525522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b26645-0a9f-4be3-b7d6-0ec1805bc06d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:34.525823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b26645-0a9f-4be3-b7d6-0ec1805bc06d None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:34.526442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b26645-0a9f-4be3-b7d6-0ec1805bc06d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:34.526741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b26645-0a9f-4be3-b7d6-0ec1805bc06d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:34.528097 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b26645-0a9f-4be3-b7d6-0ec1805bc06d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:34.532737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 450/887] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 08:33:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:34.533347 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b26645-0a9f-4be3-b7d6-0ec1805bc06d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:34.544988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 438/888] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:34.546137 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764cfad-0e82-47fc-9b38-fc9c1507f05a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:34.591210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:34.646493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b26645-0a9f-4be3-b7d6-0ec1805bc06d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:34.652598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:34.656794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764cfad-0e82-47fc-9b38-fc9c1507f05a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:34.683305 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069bb60-e980-48df-9bdc-f35b280ab44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:34.684454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069bb60-e980-48df-9bdc-f35b280ab44b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:34.685316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069bb60-e980-48df-9bdc-f35b280ab44b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:34.686049 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069bb60-e980-48df-9bdc-f35b280ab44b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:34.687038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069bb60-e980-48df-9bdc-f35b280ab44b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:34.689281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069bb60-e980-48df-9bdc-f35b280ab44b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-c069bb60-e980-48df-9bdc-f35b280ab44b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:34.699879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:34.700831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:33:34.701650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:34.702441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a74b9-8469-4a80-8036-cd7bf439ef47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:34.702441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a74b9-8469-4a80-8036-cd7bf439ef47 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:34.702441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a74b9-8469-4a80-8036-cd7bf439ef47 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:34.702441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a74b9-8469-4a80-8036-cd7bf439ef47 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:34.702716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a74b9-8469-4a80-8036-cd7bf439ef47 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:34.702716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a74b9-8469-4a80-8036-cd7bf439ef47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:34.706266 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 451/889] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:33:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:34.706430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a74b9-8469-4a80-8036-cd7bf439ef47 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:34.706516 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069bb60-e980-48df-9bdc-f35b280ab44b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:34.712349 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 439/890] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 08:33:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:34.713982 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a74b9-8469-4a80-8036-cd7bf439ef47 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:34.815805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069bb60-e980-48df-9bdc-f35b280ab44b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:34.820834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a74b9-8469-4a80-8036-cd7bf439ef47 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:34.829742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:34.850874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4783f1-9da3-4e31-990e-8e2481ab99cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:34.851471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4783f1-9da3-4e31-990e-8e2481ab99cb None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:34.851748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4783f1-9da3-4e31-990e-8e2481ab99cb None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:34.852022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4783f1-9da3-4e31-990e-8e2481ab99cb None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:34.852648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4783f1-9da3-4e31-990e-8e2481ab99cb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:34.852973 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4783f1-9da3-4e31-990e-8e2481ab99cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:34.854633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4783f1-9da3-4e31-990e-8e2481ab99cb None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:34.856688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48a5fc-c6cf-4fcb-a88e-646a00272dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:34.857286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48a5fc-c6cf-4fcb-a88e-646a00272dd1 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:34.857564 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48a5fc-c6cf-4fcb-a88e-646a00272dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:34.857864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48a5fc-c6cf-4fcb-a88e-646a00272dd1 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:34.858475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48a5fc-c6cf-4fcb-a88e-646a00272dd1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:34.858774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48a5fc-c6cf-4fcb-a88e-646a00272dd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:34.858995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 452/891] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 08:33:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:34.859762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4783f1-9da3-4e31-990e-8e2481ab99cb None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:34.860099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48a5fc-c6cf-4fcb-a88e-646a00272dd1 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:34.876473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 440/892] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:33:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:34.876473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48a5fc-c6cf-4fcb-a88e-646a00272dd1 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:34.913098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:34.971108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4783f1-9da3-4e31-990e-8e2481ab99cb None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:34.993189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48a5fc-c6cf-4fcb-a88e-646a00272dd1 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:35.012821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb053ac-41bd-427b-b425-d51d1373d93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:35.013448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb053ac-41bd-427b-b425-d51d1373d93e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:35.013750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb053ac-41bd-427b-b425-d51d1373d93e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:35.014023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb053ac-41bd-427b-b425-d51d1373d93e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:35.014736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb053ac-41bd-427b-b425-d51d1373d93e None admin] RBAC: Adding query filter params (interface=public, service_id=928523e096914b44a5caced9f284b937, region_id=RegionOne) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:35.015041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb053ac-41bd-427b-b425-d51d1373d93e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:35.017229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb053ac-41bd-427b-b425-d51d1373d93e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:35.028102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 453/893] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 08:33:34 2022] GET /identity/v3/endpoints?service_id=928523e096914b44a5caced9f284b937&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:35.029038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb053ac-41bd-427b-b425-d51d1373d93e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:35.031480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22ffb8-9776-4c0a-8dbb-7916c086e9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:35.032117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22ffb8-9776-4c0a-8dbb-7916c086e9a6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:35.032398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22ffb8-9776-4c0a-8dbb-7916c086e9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:35.032673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22ffb8-9776-4c0a-8dbb-7916c086e9a6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:35.045209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22ffb8-9776-4c0a-8dbb-7916c086e9a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-7d22ffb8-9776-4c0a-8dbb-7916c086e9a6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:35.050623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:35.051610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 441/894] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Oct 4 08:33:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:35.052822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22ffb8-9776-4c0a-8dbb-7916c086e9a6 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:35.079607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:35.140400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb053ac-41bd-427b-b425-d51d1373d93e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:35.143550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:35.157086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22ffb8-9776-4c0a-8dbb-7916c086e9a6 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:35.177520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ce41f-9bc1-49df-bac6-3f6fe90b87d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:35.178174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ce41f-9bc1-49df-bac6-3f6fe90b87d0 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:35.178437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ce41f-9bc1-49df-bac6-3f6fe90b87d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:35.178717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ce41f-9bc1-49df-bac6-3f6fe90b87d0 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:35.179425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ce41f-9bc1-49df-bac6-3f6fe90b87d0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:35.179658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ce41f-9bc1-49df-bac6-3f6fe90b87d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:35.181029 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ce41f-9bc1-49df-bac6-3f6fe90b87d0 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:35.186101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 454/895] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:33:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:35.190713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ce41f-9bc1-49df-bac6-3f6fe90b87d0 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:35.192070 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee39b4c-e985-43f1-9211-7b84050d3ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:35.193202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee39b4c-e985-43f1-9211-7b84050d3ae5 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:35.193479 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee39b4c-e985-43f1-9211-7b84050d3ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:35.193779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee39b4c-e985-43f1-9211-7b84050d3ae5 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:35.194379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee39b4c-e985-43f1-9211-7b84050d3ae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:35.195737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee39b4c-e985-43f1-9211-7b84050d3ae5 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:35.210186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 442/896] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 08:33:35 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:33:35.212332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee39b4c-e985-43f1-9211-7b84050d3ae5 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:35.313118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ce41f-9bc1-49df-bac6-3f6fe90b87d0 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:35.323964 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:35.354720 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22212a8-81a2-4ff2-b9b9-00afe0b52387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:35.355341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22212a8-81a2-4ff2-b9b9-00afe0b52387 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:35.355622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22212a8-81a2-4ff2-b9b9-00afe0b52387 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:35.355891 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22212a8-81a2-4ff2-b9b9-00afe0b52387 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:35.356563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22212a8-81a2-4ff2-b9b9-00afe0b52387 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:35.356866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22212a8-81a2-4ff2-b9b9-00afe0b52387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:35.358306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22212a8-81a2-4ff2-b9b9-00afe0b52387 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:35.368352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 455/897] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 08:33:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:35.375425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22212a8-81a2-4ff2-b9b9-00afe0b52387 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:35.377518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee39b4c-e985-43f1-9211-7b84050d3ae5 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:35.419792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bd733-d8fb-4de9-9a66-9c38375ca89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:35.420399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bd733-d8fb-4de9-9a66-9c38375ca89e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:35.420680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bd733-d8fb-4de9-9a66-9c38375ca89e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:35.420958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20bd733-d8fb-4de9-9a66-9c38375ca89e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:35.421657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bd733-d8fb-4de9-9a66-9c38375ca89e None admin] RBAC: Adding query filter params (interface=public, service_id=fc532dbbcb5b44059d0b2d0bbe2e47f0, region_id=RegionOne) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:35.421992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bd733-d8fb-4de9-9a66-9c38375ca89e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:35.423350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20bd733-d8fb-4de9-9a66-9c38375ca89e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:35.429146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 443/898] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 08:33:35 2022] GET /identity/v3/endpoints?service_id=fc532dbbcb5b44059d0b2d0bbe2e47f0&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:35.434664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bd733-d8fb-4de9-9a66-9c38375ca89e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:35.452897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:35.506202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22212a8-81a2-4ff2-b9b9-00afe0b52387 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:35.542159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9952861-6235-46ea-b9b1-c7ec5a43e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:35.542789 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9952861-6235-46ea-b9b1-c7ec5a43e2f2 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:35.543069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9952861-6235-46ea-b9b1-c7ec5a43e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:35.543343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9952861-6235-46ea-b9b1-c7ec5a43e2f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:35.548821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20bd733-d8fb-4de9-9a66-9c38375ca89e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:35.554931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9952861-6235-46ea-b9b1-c7ec5a43e2f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:35.556448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9952861-6235-46ea-b9b1-c7ec5a43e2f2 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-d9952861-6235-46ea-b9b1-c7ec5a43e2f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:35.571445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:35.572494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:35.573424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:35.573424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:35.573424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:35.573424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 456/899] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:35.579375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9952861-6235-46ea-b9b1-c7ec5a43e2f2 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:35.590655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38264e-004d-4a55-bb11-8d26ba80b760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:35.591267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38264e-004d-4a55-bb11-8d26ba80b760 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:35.591549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38264e-004d-4a55-bb11-8d26ba80b760 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:35.591822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38264e-004d-4a55-bb11-8d26ba80b760 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:35.592423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38264e-004d-4a55-bb11-8d26ba80b760 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:35.593943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38264e-004d-4a55-bb11-8d26ba80b760 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:35.600460 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 444/900] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 08:33:35 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:33:35.628751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:35.713678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9952861-6235-46ea-b9b1-c7ec5a43e2f2 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:35.751592 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61e6fa-8bd4-46d2-bff2-c976b9913d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:35.752277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61e6fa-8bd4-46d2-bff2-c976b9913d6b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:35.752562 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61e6fa-8bd4-46d2-bff2-c976b9913d6b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:35.752838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61e6fa-8bd4-46d2-bff2-c976b9913d6b None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:35.753497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61e6fa-8bd4-46d2-bff2-c976b9913d6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:35.753834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61e6fa-8bd4-46d2-bff2-c976b9913d6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:35.755289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61e6fa-8bd4-46d2-bff2-c976b9913d6b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:35.768279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 457/901] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:35.775047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38264e-004d-4a55-bb11-8d26ba80b760 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:35.863598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:35.883858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38264e-004d-4a55-bb11-8d26ba80b760 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:35.918787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aec593-88c0-4b33-8f83-b59c1434615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:35.919418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aec593-88c0-4b33-8f83-b59c1434615f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:35.919694 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aec593-88c0-4b33-8f83-b59c1434615f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:35.919964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aec593-88c0-4b33-8f83-b59c1434615f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:35.927315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aec593-88c0-4b33-8f83-b59c1434615f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:35.928760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aec593-88c0-4b33-8f83-b59c1434615f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-87aec593-88c0-4b33-8f83-b59c1434615f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:35.940379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:35.943037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:35.945164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:35.945164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:35.945164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:35.945164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:35.945164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:35.945164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:35.945164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:35.945164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:35.945164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:33:35.945164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:35.945164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 445/902] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 08:33:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:35.955648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61e6fa-8bd4-46d2-bff2-c976b9913d6b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:36.041775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:36.061193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61e6fa-8bd4-46d2-bff2-c976b9913d6b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:36.100020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973c25f-01c8-40e9-b169-16bbd63129ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:36.100624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973c25f-01c8-40e9-b169-16bbd63129ae None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:36.100898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973c25f-01c8-40e9-b169-16bbd63129ae None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:36.101169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973c25f-01c8-40e9-b169-16bbd63129ae None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:36.101915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973c25f-01c8-40e9-b169-16bbd63129ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:36.102220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973c25f-01c8-40e9-b169-16bbd63129ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:36.103647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973c25f-01c8-40e9-b169-16bbd63129ae None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:36.121660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 458/903] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Oct 4 08:33:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:36.126960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aec593-88c0-4b33-8f83-b59c1434615f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:36.233804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aec593-88c0-4b33-8f83-b59c1434615f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:36.268253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1253b-e71d-434c-a1d0-cb321c4aae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:36.268858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1253b-e71d-434c-a1d0-cb321c4aae09 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:36.269185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1253b-e71d-434c-a1d0-cb321c4aae09 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:36.269462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1253b-e71d-434c-a1d0-cb321c4aae09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:36.286607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:36.339471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1253b-e71d-434c-a1d0-cb321c4aae09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:36.340992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1253b-e71d-434c-a1d0-cb321c4aae09 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:36.442949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 446/904] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:36 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:33:36.448660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973c25f-01c8-40e9-b169-16bbd63129ae None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:36.550852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973c25f-01c8-40e9-b169-16bbd63129ae None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:36.586368 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4877b-04a9-4d56-af80-b9db28012240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:36.586986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4877b-04a9-4d56-af80-b9db28012240 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:36.587261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4877b-04a9-4d56-af80-b9db28012240 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:36.587528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4877b-04a9-4d56-af80-b9db28012240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:36.594267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:36.599239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4877b-04a9-4d56-af80-b9db28012240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:36.601353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4877b-04a9-4d56-af80-b9db28012240 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-d0c4877b-04a9-4d56-af80-b9db28012240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:36.617273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:36.618356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:36.619317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:36.619317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:36.619317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:36.619317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 459/905] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:36.624582 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1253b-e71d-434c-a1d0-cb321c4aae09 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:36.681373 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:36.741132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1253b-e71d-434c-a1d0-cb321c4aae09 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:36.776093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a1141-e4ad-474b-8067-e4405d209418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:36.776900 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a1141-e4ad-474b-8067-e4405d209418 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:36.777342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a1141-e4ad-474b-8067-e4405d209418 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:36.777795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a1141-e4ad-474b-8067-e4405d209418 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:36.778589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a1141-e4ad-474b-8067-e4405d209418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:36.779059 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a1141-e4ad-474b-8067-e4405d209418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:36.780559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a1141-e4ad-474b-8067-e4405d209418 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:36.792845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 447/906] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:36.798559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4877b-04a9-4d56-af80-b9db28012240 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:36.899229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4877b-04a9-4d56-af80-b9db28012240 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:36.933722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d772fe9-b1ad-4c99-ac77-3b99725629d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:36.934607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d772fe9-b1ad-4c99-ac77-3b99725629d5 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:36.935047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d772fe9-b1ad-4c99-ac77-3b99725629d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:36.935472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d772fe9-b1ad-4c99-ac77-3b99725629d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:36.946469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d772fe9-b1ad-4c99-ac77-3b99725629d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:36.947928 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d772fe9-b1ad-4c99-ac77-3b99725629d5 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-1d772fe9-b1ad-4c99-ac77-3b99725629d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:36.959866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:36.961206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:36.962130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:36.962130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:36.962130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:36.962130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:36.962130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:36.962130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:36.962130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:36.962130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:36.962130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 460/907] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:36.969949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a1141-e4ad-474b-8067-e4405d209418 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:37.075113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:37.078234 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a1141-e4ad-474b-8067-e4405d209418 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:37.114224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37847c-0a5c-4eb0-bec5-1d57c3a7926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:37.114893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37847c-0a5c-4eb0-bec5-1d57c3a7926b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:37.115171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37847c-0a5c-4eb0-bec5-1d57c3a7926b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:37.115440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37847c-0a5c-4eb0-bec5-1d57c3a7926b None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:37.116173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37847c-0a5c-4eb0-bec5-1d57c3a7926b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:37.116366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27f323-a45e-4861-991e-7f9d2fc7ca95 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:37.116458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37847c-0a5c-4eb0-bec5-1d57c3a7926b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:37.116632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27f323-a45e-4861-991e-7f9d2fc7ca95 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:37.116896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27f323-a45e-4861-991e-7f9d2fc7ca95 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:37.117907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37847c-0a5c-4eb0-bec5-1d57c3a7926b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:37.119682 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 461/908] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:37.123683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4375c-c06c-42ab-b75a-d12d9500f0f3 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:37.123975 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4375c-c06c-42ab-b75a-d12d9500f0f3 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:37.124255 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4375c-c06c-42ab-b75a-d12d9500f0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:37.131108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 448/909] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:37.138672 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37847c-0a5c-4eb0-bec5-1d57c3a7926b None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:37.142938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-c9e4375c-c06c-42ab-b75a-d12d9500f0f3 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:37.144148 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c9e4375c-c06c-42ab-b75a-d12d9500f0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:37.193448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:37.225945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 462/910] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:37.253726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37847c-0a5c-4eb0-bec5-1d57c3a7926b None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:37.289263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99af95-24f6-43f1-ae06-777b6893bdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:37.290299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99af95-24f6-43f1-ae06-777b6893bdb6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:37.290818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99af95-24f6-43f1-ae06-777b6893bdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:37.291329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99af95-24f6-43f1-ae06-777b6893bdb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:37.292286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99af95-24f6-43f1-ae06-777b6893bdb6 None admin] RBAC: Adding query filter params (role.id=9b17208db146464c84a15bc97da278d9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=595dac02b846405cab34dcf898ce5b5a) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:37.292810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99af95-24f6-43f1-ae06-777b6893bdb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:37.294575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99af95-24f6-43f1-ae06-777b6893bdb6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:37.300434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 449/911] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:37 2022] GET /identity/v3/role_assignments?user.id=595dac02b846405cab34dcf898ce5b5a&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=9b17208db146464c84a15bc97da278d9 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:37.512760 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4375c-c06c-42ab-b75a-d12d9500f0f3 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:37.528689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590aed0-befe-4d9d-b4f2-c227f6ab7fdb None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:37.529039 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590aed0-befe-4d9d-b4f2-c227f6ab7fdb None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:37.529234 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590aed0-befe-4d9d-b4f2-c227f6ab7fdb None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:37.530837 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 450/912] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:33:37.535777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b79c6-5e9c-4764-b9d8-a67dc4c369d3 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:37.536090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b79c6-5e9c-4764-b9d8-a67dc4c369d3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:37.536363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b79c6-5e9c-4764-b9d8-a67dc4c369d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:37.554079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-e66b79c6-5e9c-4764-b9d8-a67dc4c369d3 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:37.555260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e66b79c6-5e9c-4764-b9d8-a67dc4c369d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:37.577351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:37.628429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4375c-c06c-42ab-b75a-d12d9500f0f3 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:37.651985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 451/913] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:37.674669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb274cb6-c8db-4f06-962c-9e70892c5d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:37.675609 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb274cb6-c8db-4f06-962c-9e70892c5d00 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:37.676110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb274cb6-c8db-4f06-962c-9e70892c5d00 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:37.677136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb274cb6-c8db-4f06-962c-9e70892c5d00 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:37.678050 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb274cb6-c8db-4f06-962c-9e70892c5d00 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:37.679622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb274cb6-c8db-4f06-962c-9e70892c5d00 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:37.696914 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 463/914] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:33:37 2022] POST /identity/v3/services => generated 241 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:33:37.916159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b79c6-5e9c-4764-b9d8-a67dc4c369d3 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:38.015462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b79c6-5e9c-4764-b9d8-a67dc4c369d3 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:38.024416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4727787-4f06-44a7-8804-5feed3f0644f None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:38.024863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4727787-4f06-44a7-8804-5feed3f0644f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:38.024950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4727787-4f06-44a7-8804-5feed3f0644f None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:38.026535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 464/915] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:38.031845 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81310c09-ca5c-42a0-a6c7-71427e466310 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:38.032094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81310c09-ca5c-42a0-a6c7-71427e466310 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:38.032362 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81310c09-ca5c-42a0-a6c7-71427e466310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:38.052274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-81310c09-ca5c-42a0-a6c7-71427e466310 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:38.053459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-81310c09-ca5c-42a0-a6c7-71427e466310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:38.059260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052c2d8-0fa8-4e5e-96d1-e102dc66cdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:38.059893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052c2d8-0fa8-4e5e-96d1-e102dc66cdac None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:38.060169 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052c2d8-0fa8-4e5e-96d1-e102dc66cdac None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:38.060436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052c2d8-0fa8-4e5e-96d1-e102dc66cdac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:38.061005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052c2d8-0fa8-4e5e-96d1-e102dc66cdac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:38.062386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052c2d8-0fa8-4e5e-96d1-e102dc66cdac None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-f052c2d8-0fa8-4e5e-96d1-e102dc66cdac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:38.074241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:38.075275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:38.076179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 452/916] 10.222.0.26 () {56 vars in 1132 bytes} [Tue Oct 4 08:33:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:38.081092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052c2d8-0fa8-4e5e-96d1-e102dc66cdac None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:38.098061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:38.099800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:38.139951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 465/917] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:38.193717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052c2d8-0fa8-4e5e-96d1-e102dc66cdac None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:38.229262 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e0566-b8fd-479e-baf6-8c1f31de7d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:38.230240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e0566-b8fd-479e-baf6-8c1f31de7d4a None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:38.230714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e0566-b8fd-479e-baf6-8c1f31de7d4a None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:38.231171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e0566-b8fd-479e-baf6-8c1f31de7d4a None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:38.231966 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e0566-b8fd-479e-baf6-8c1f31de7d4a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:38.232453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e0566-b8fd-479e-baf6-8c1f31de7d4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:38.234041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e0566-b8fd-479e-baf6-8c1f31de7d4a None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:38.239982 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 453/918] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 08:33:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:38.248591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81310c09-ca5c-42a0-a6c7-71427e466310 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:38.351327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81310c09-ca5c-42a0-a6c7-71427e466310 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:38.386359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaedf5-dbd3-43f7-9e49-4bbc2382e16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:38.387335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaedf5-dbd3-43f7-9e49-4bbc2382e16f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:38.387883 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaedf5-dbd3-43f7-9e49-4bbc2382e16f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:38.388348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eaedf5-dbd3-43f7-9e49-4bbc2382e16f None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:38.389179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eaedf5-dbd3-43f7-9e49-4bbc2382e16f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:38.389709 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eaedf5-dbd3-43f7-9e49-4bbc2382e16f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:38.391359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eaedf5-dbd3-43f7-9e49-4bbc2382e16f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:38.396777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 466/919] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 08:33:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:38.402747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e0566-b8fd-479e-baf6-8c1f31de7d4a None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:38.442262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaedf5-dbd3-43f7-9e49-4bbc2382e16f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:38.479868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:38.498538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:38.519406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e0566-b8fd-479e-baf6-8c1f31de7d4a None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:38.554915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eaedf5-dbd3-43f7-9e49-4bbc2382e16f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:38.556694 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002bd1-b4bb-4370-b44b-201d5966603e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:38.557317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002bd1-b4bb-4370-b44b-201d5966603e None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:38.557598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002bd1-b4bb-4370-b44b-201d5966603e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:38.557902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002bd1-b4bb-4370-b44b-201d5966603e None admin] PATH_INFO: `/v3/endpoints` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:38.559087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d002bd1-b4bb-4370-b44b-201d5966603e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:38.560056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d002bd1-b4bb-4370-b44b-201d5966603e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:38.590367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 454/920] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:33:38 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:33:38.612931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb3c8b-a905-4c9c-9746-7a5f4edfa14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:38.613577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb3c8b-a905-4c9c-9746-7a5f4edfa14c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:38.614469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb3c8b-a905-4c9c-9746-7a5f4edfa14c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:38.614754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb3c8b-a905-4c9c-9746-7a5f4edfa14c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:38.615404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb3c8b-a905-4c9c-9746-7a5f4edfa14c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:38.616835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb3c8b-a905-4c9c-9746-7a5f4edfa14c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-abcb3c8b-a905-4c9c-9746-7a5f4edfa14c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:38.627286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:38.628345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:38.629185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 467/921] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:33:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:38.634584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002bd1-b4bb-4370-b44b-201d5966603e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:38.761196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002bd1-b4bb-4370-b44b-201d5966603e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:38.796894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009ccab-6bf2-4e29-a7c8-8ac26e46c5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:38.797506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009ccab-6bf2-4e29-a7c8-8ac26e46c5e3 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:38.797823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009ccab-6bf2-4e29-a7c8-8ac26e46c5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:38.798094 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009ccab-6bf2-4e29-a7c8-8ac26e46c5e3 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:38.798717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009ccab-6bf2-4e29-a7c8-8ac26e46c5e3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:38.799045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009ccab-6bf2-4e29-a7c8-8ac26e46c5e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:38.800402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009ccab-6bf2-4e29-a7c8-8ac26e46c5e3 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:38.801651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:38.806664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 455/922] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:33:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:38.812022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb3c8b-a905-4c9c-9746-7a5f4edfa14c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:38.913110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:38.921247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb3c8b-a905-4c9c-9746-7a5f4edfa14c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:38.961717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969947a-b052-4bd9-aad6-536c9b0eac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:38.962329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969947a-b052-4bd9-aad6-536c9b0eac31 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:38.962613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969947a-b052-4bd9-aad6-536c9b0eac31 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:38.962887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969947a-b052-4bd9-aad6-536c9b0eac31 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:38.963525 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969947a-b052-4bd9-aad6-536c9b0eac31 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:38.963837 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969947a-b052-4bd9-aad6-536c9b0eac31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:38.965206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969947a-b052-4bd9-aad6-536c9b0eac31 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:38.970503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 468/923] 10.222.0.26 () {56 vars in 1114 bytes} [Tue Oct 4 08:33:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:38.976464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009ccab-6bf2-4e29-a7c8-8ac26e46c5e3 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:39.085720 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009ccab-6bf2-4e29-a7c8-8ac26e46c5e3 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:39.125927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f872fd-ce42-4f3c-b4c8-6313827dc7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:39.126544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f872fd-ce42-4f3c-b4c8-6313827dc7f8 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:39.126826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f872fd-ce42-4f3c-b4c8-6313827dc7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:39.127103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f872fd-ce42-4f3c-b4c8-6313827dc7f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:39.127783 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f872fd-ce42-4f3c-b4c8-6313827dc7f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:39.129149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f872fd-ce42-4f3c-b4c8-6313827dc7f8 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:39.151447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 456/924] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:33:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:33:39.527066 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b36837-3ee8-415f-8530-fd9adc6ea9d3 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:39.527599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b36837-3ee8-415f-8530-fd9adc6ea9d3 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:39.527599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b36837-3ee8-415f-8530-fd9adc6ea9d3 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:39.529137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 469/925] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:39.534593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca347cf-8ad9-4e5a-99cf-aca38e675512 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:39.535133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca347cf-8ad9-4e5a-99cf-aca38e675512 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:39.535620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca347cf-8ad9-4e5a-99cf-aca38e675512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:39.554725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-0ca347cf-8ad9-4e5a-99cf-aca38e675512 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:39.556252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0ca347cf-8ad9-4e5a-99cf-aca38e675512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:39.609279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:39.636546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 457/926] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:39.906354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b36837-3ee8-415f-8530-fd9adc6ea9d3 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:40.020580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b36837-3ee8-415f-8530-fd9adc6ea9d3 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:40.044735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff7ccc-aab3-4a4e-91c6-26a2d12bd53a None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:40.045355 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff7ccc-aab3-4a4e-91c6-26a2d12bd53a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:40.045877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff7ccc-aab3-4a4e-91c6-26a2d12bd53a None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:40.047636 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 458/927] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:40.053448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398df9fb-bda4-469c-9220-db1db95be1eb None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:40.054075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398df9fb-bda4-469c-9220-db1db95be1eb None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:40.054574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398df9fb-bda4-469c-9220-db1db95be1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:40.057780 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b64459-77b6-4446-8a97-e3022e02f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:40.058549 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b64459-77b6-4446-8a97-e3022e02f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:40.058842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b64459-77b6-4446-8a97-e3022e02f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:40.059123 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b64459-77b6-4446-8a97-e3022e02f9f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:40.072267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b64459-77b6-4446-8a97-e3022e02f9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:40.075135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-398df9fb-bda4-469c-9220-db1db95be1eb None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:40.076632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-398df9fb-bda4-469c-9220-db1db95be1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-94b64459-77b6-4446-8a97-e3022e02f9f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:40.077300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:40.078269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:40.079100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 470/928] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Oct 4 08:33:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:40.086181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b64459-77b6-4446-8a97-e3022e02f9f0 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:40.110006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:40.173828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 459/929] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:40.205567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b64459-77b6-4446-8a97-e3022e02f9f0 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:40.242206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ac1b1-8381-48b5-bf69-5ec389d0dc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:40.243067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ac1b1-8381-48b5-bf69-5ec389d0dc51 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:40.243494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ac1b1-8381-48b5-bf69-5ec389d0dc51 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:40.243924 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ac1b1-8381-48b5-bf69-5ec389d0dc51 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:40.245099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ac1b1-8381-48b5-bf69-5ec389d0dc51 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:40.246041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ac1b1-8381-48b5-bf69-5ec389d0dc51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:40.248063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ac1b1-8381-48b5-bf69-5ec389d0dc51 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:40.256908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 471/930] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 08:33:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:40.265734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398df9fb-bda4-469c-9220-db1db95be1eb None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:40.364110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398df9fb-bda4-469c-9220-db1db95be1eb None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:40.374948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:40.400519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c805d-91af-4b7d-b819-0445917868fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:40.401159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c805d-91af-4b7d-b819-0445917868fd None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:40.401443 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c805d-91af-4b7d-b819-0445917868fd None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:40.401739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c805d-91af-4b7d-b819-0445917868fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:40.412178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c805d-91af-4b7d-b819-0445917868fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:40.413631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c805d-91af-4b7d-b819-0445917868fd None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-474c805d-91af-4b7d-b819-0445917868fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:40.429294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:40.430367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:40.431277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:40.431277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:40.431277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:40.431277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 460/931] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:40.435994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ac1b1-8381-48b5-bf69-5ec389d0dc51 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:40.441887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c805d-91af-4b7d-b819-0445917868fd None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:40.527870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:40.548748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ac1b1-8381-48b5-bf69-5ec389d0dc51 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:40.549384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c805d-91af-4b7d-b819-0445917868fd None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:40.584077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885f612-07a0-4dde-8fb0-4d3dda2ee33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:40.584561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33400c5-cade-4600-826a-c29db7baaf6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:40.584696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885f612-07a0-4dde-8fb0-4d3dda2ee33e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:40.584951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885f612-07a0-4dde-8fb0-4d3dda2ee33e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:40.585185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33400c5-cade-4600-826a-c29db7baaf6f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:40.585296 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885f612-07a0-4dde-8fb0-4d3dda2ee33e None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:40.585470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33400c5-cade-4600-826a-c29db7baaf6f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:40.585829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33400c5-cade-4600-826a-c29db7baaf6f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:40.585932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885f612-07a0-4dde-8fb0-4d3dda2ee33e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:40.586193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885f612-07a0-4dde-8fb0-4d3dda2ee33e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:40.586299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33400c5-cade-4600-826a-c29db7baaf6f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:40.587560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885f612-07a0-4dde-8fb0-4d3dda2ee33e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:40.587670 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33400c5-cade-4600-826a-c29db7baaf6f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-a33400c5-cade-4600-826a-c29db7baaf6f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:40.596666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:40.597666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:40.599103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 461/932] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 08:33:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:40.606281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 472/933] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:40.606682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33400c5-cade-4600-826a-c29db7baaf6f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:40.612736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885f612-07a0-4dde-8fb0-4d3dda2ee33e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:40.636660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:40.714142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33400c5-cade-4600-826a-c29db7baaf6f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:40.727169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885f612-07a0-4dde-8fb0-4d3dda2ee33e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:40.745281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d845-75d1-4b14-be56-3975de7a1e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:40.746064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d845-75d1-4b14-be56-3975de7a1e88 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:40.746405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d845-75d1-4b14-be56-3975de7a1e88 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:40.746747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d845-75d1-4b14-be56-3975de7a1e88 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:40.747449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929d845-75d1-4b14-be56-3975de7a1e88 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:40.747829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929d845-75d1-4b14-be56-3975de7a1e88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:40.749259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929d845-75d1-4b14-be56-3975de7a1e88 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:40.754777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 462/934] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 08:33:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:40.763403 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d845-75d1-4b14-be56-3975de7a1e88 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:40.771595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad054fd-b51f-47a0-b600-0fd25b18aade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:40.772435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad054fd-b51f-47a0-b600-0fd25b18aade None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:40.772877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad054fd-b51f-47a0-b600-0fd25b18aade None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:40.773300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad054fd-b51f-47a0-b600-0fd25b18aade None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:40.781428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad054fd-b51f-47a0-b600-0fd25b18aade None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:40.783188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad054fd-b51f-47a0-b600-0fd25b18aade None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:40.787445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-4ad054fd-b51f-47a0-b600-0fd25b18aade None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:40.799902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:40.800882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:40.801893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:40.801893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:40.801893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:40.801893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:40.801893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:40.801893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:40.801893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:40.801893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:40.801893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:33:40.801893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:40.804475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 473/935] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 08:33:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:40.810198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad054fd-b51f-47a0-b600-0fd25b18aade None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:40.877128 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:40.888982 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d845-75d1-4b14-be56-3975de7a1e88 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:40.929212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4dea0-7f5e-48ce-8f89-6fbdba1ec520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:40.930262 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4dea0-7f5e-48ce-8f89-6fbdba1ec520 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:40.930727 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4dea0-7f5e-48ce-8f89-6fbdba1ec520 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:40.931205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4dea0-7f5e-48ce-8f89-6fbdba1ec520 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:40.932037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4dea0-7f5e-48ce-8f89-6fbdba1ec520 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:40.932534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4dea0-7f5e-48ce-8f89-6fbdba1ec520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:40.934203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4dea0-7f5e-48ce-8f89-6fbdba1ec520 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:40.935105 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad054fd-b51f-47a0-b600-0fd25b18aade None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:40.946120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 463/936] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 08:33:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:40.947715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4dea0-7f5e-48ce-8f89-6fbdba1ec520 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:40.977948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a74b76-e577-4cc3-8514-5091cf70adc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:40.978491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a74b76-e577-4cc3-8514-5091cf70adc6 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:40.979197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a74b76-e577-4cc3-8514-5091cf70adc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:40.979481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a74b76-e577-4cc3-8514-5091cf70adc6 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:40.980536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a74b76-e577-4cc3-8514-5091cf70adc6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:40.980853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a74b76-e577-4cc3-8514-5091cf70adc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:40.982571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a74b76-e577-4cc3-8514-5091cf70adc6 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:41.005158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 474/937] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Oct 4 08:33:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:41.010844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a74b76-e577-4cc3-8514-5091cf70adc6 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:41.059557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4dea0-7f5e-48ce-8f89-6fbdba1ec520 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:41.076155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:41.099960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822789dc-ae7c-4681-a811-5555f083d084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:41.100525 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822789dc-ae7c-4681-a811-5555f083d084 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:41.100809 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822789dc-ae7c-4681-a811-5555f083d084 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:41.101358 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822789dc-ae7c-4681-a811-5555f083d084 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:41.101808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822789dc-ae7c-4681-a811-5555f083d084 None admin] RBAC: Adding query filter params (interface=public, service_id=94dfb87bd7204b0798c8c0d6e717b4e5, region_id=RegionOne) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:41.102112 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822789dc-ae7c-4681-a811-5555f083d084 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:41.103487 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822789dc-ae7c-4681-a811-5555f083d084 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:41.110423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 464/938] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 08:33:40 2022] GET /identity/v3/endpoints?service_id=94dfb87bd7204b0798c8c0d6e717b4e5&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:41.111607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f967bf5-4685-4aed-8881-501b7b3b27a3 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:41.111895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f967bf5-4685-4aed-8881-501b7b3b27a3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:41.112174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f967bf5-4685-4aed-8881-501b7b3b27a3 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:41.118038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 465/939] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:41.118830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f967bf5-4685-4aed-8881-501b7b3b27a3 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:41.125597 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a74b76-e577-4cc3-8514-5091cf70adc6 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:41.137802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:41.164014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb691b78-5d7f-4759-b998-5cecccba75b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:41.164720 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb691b78-5d7f-4759-b998-5cecccba75b1 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:41.164921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb691b78-5d7f-4759-b998-5cecccba75b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:41.165199 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb691b78-5d7f-4759-b998-5cecccba75b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:41.175548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb691b78-5d7f-4759-b998-5cecccba75b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:41.177593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb691b78-5d7f-4759-b998-5cecccba75b1 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-cb691b78-5d7f-4759-b998-5cecccba75b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:41.192469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:41.193631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:41.194677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:41.194677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:41.194677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:41.194677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 475/940] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:41.195550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba9035-c41c-4160-8fe8-454f568b3a59 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:41.195550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba9035-c41c-4160-8fe8-454f568b3a59 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:41.195733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba9035-c41c-4160-8fe8-454f568b3a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:41.213476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d8ba9035-c41c-4160-8fe8-454f568b3a59 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:41.214746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d8ba9035-c41c-4160-8fe8-454f568b3a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:41.233726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f967bf5-4685-4aed-8881-501b7b3b27a3 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:41.276545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9aea12-6d38-4355-8ecf-16975672e735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:41.277170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9aea12-6d38-4355-8ecf-16975672e735 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:41.277454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9aea12-6d38-4355-8ecf-16975672e735 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:41.277757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9aea12-6d38-4355-8ecf-16975672e735 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:41.278365 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9aea12-6d38-4355-8ecf-16975672e735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:41.279773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9aea12-6d38-4355-8ecf-16975672e735 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:41.286139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 466/941] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 08:33:41 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:33:41.286901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9aea12-6d38-4355-8ecf-16975672e735 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:41.300658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 476/942] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:41.371655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404a7d1-c06f-47bc-ac04-07a26a5f6a37 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:41.372001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404a7d1-c06f-47bc-ac04-07a26a5f6a37 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:41.372225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2404a7d1-c06f-47bc-ac04-07a26a5f6a37 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:41.373883 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 477/943] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:33:41.379334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289f992-caf4-457c-b094-71b7ef1f52aa None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:41.380169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289f992-caf4-457c-b094-71b7ef1f52aa None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:41.380460 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289f992-caf4-457c-b094-71b7ef1f52aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:41.392670 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:41.400245 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-8289f992-caf4-457c-b094-71b7ef1f52aa None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:41.401919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8289f992-caf4-457c-b094-71b7ef1f52aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:41.413647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9aea12-6d38-4355-8ecf-16975672e735 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:41.448833 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:41.459765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a79e3e-c71e-40a3-8f73-e89e9a253acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:41.460775 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a79e3e-c71e-40a3-8f73-e89e9a253acc None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:41.461282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a79e3e-c71e-40a3-8f73-e89e9a253acc None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:41.461808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a79e3e-c71e-40a3-8f73-e89e9a253acc None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:41.462710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a79e3e-c71e-40a3-8f73-e89e9a253acc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:41.463271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a79e3e-c71e-40a3-8f73-e89e9a253acc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:41.465004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a79e3e-c71e-40a3-8f73-e89e9a253acc None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:41.484437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 467/944] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:41.486542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a79e3e-c71e-40a3-8f73-e89e9a253acc None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:41.494794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 478/945] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:41.584764 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289f992-caf4-457c-b094-71b7ef1f52aa None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:41.604182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a79e3e-c71e-40a3-8f73-e89e9a253acc None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:41.640584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6a5b7-73a5-4534-bc20-d8942baec547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:41.641321 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6a5b7-73a5-4534-bc20-d8942baec547 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:41.641608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6a5b7-73a5-4534-bc20-d8942baec547 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:41.641927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6a5b7-73a5-4534-bc20-d8942baec547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:41.653128 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6a5b7-73a5-4534-bc20-d8942baec547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:41.654623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6a5b7-73a5-4534-bc20-d8942baec547 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-c7a6a5b7-73a5-4534-bc20-d8942baec547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:41.672937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:41.673874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:41.674800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:41.674800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:41.674800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:41.674800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:41.674800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:41.674800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:41.674800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:41.674800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:41.674800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 468/946] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:41.685508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6a5b7-73a5-4534-bc20-d8942baec547 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:41.688713 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:41.722805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289f992-caf4-457c-b094-71b7ef1f52aa None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:41.760112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46796db1-fe13-41ce-899c-f9e8326610ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:41.761135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46796db1-fe13-41ce-899c-f9e8326610ad None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:41.761907 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46796db1-fe13-41ce-899c-f9e8326610ad None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:41.762461 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46796db1-fe13-41ce-899c-f9e8326610ad None admin] PATH_INFO: `/v3/services/compute` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:41.763318 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46796db1-fe13-41ce-899c-f9e8326610ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:41.764846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46796db1-fe13-41ce-899c-f9e8326610ad None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-46796db1-fe13-41ce-899c-f9e8326610ad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:41.777591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:41.778626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:41.779452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 479/947] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:41.780278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46796db1-fe13-41ce-899c-f9e8326610ad None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:41.808947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6a5b7-73a5-4534-bc20-d8942baec547 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:41.816654 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:41.847496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d748a3-8c27-4330-87f9-8bc09b345e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:41.848127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d748a3-8c27-4330-87f9-8bc09b345e40 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:41.848418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d748a3-8c27-4330-87f9-8bc09b345e40 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:41.848693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d748a3-8c27-4330-87f9-8bc09b345e40 None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:41.849516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d748a3-8c27-4330-87f9-8bc09b345e40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:41.849830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d748a3-8c27-4330-87f9-8bc09b345e40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:41.851366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d748a3-8c27-4330-87f9-8bc09b345e40 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:41.871385 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 469/948] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:41.871385 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d748a3-8c27-4330-87f9-8bc09b345e40 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:41.889848 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46796db1-fe13-41ce-899c-f9e8326610ad None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:41.924444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee876e-ea68-43e3-89da-3a52ddf6632c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:41.925063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee876e-ea68-43e3-89da-3a52ddf6632c None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:41.925351 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee876e-ea68-43e3-89da-3a52ddf6632c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:41.925628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee876e-ea68-43e3-89da-3a52ddf6632c None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:41.926338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee876e-ea68-43e3-89da-3a52ddf6632c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:41.927785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee876e-ea68-43e3-89da-3a52ddf6632c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:41.930398 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: WARNING keystone.server.flask.common [None req-9eee876e-ea68-43e3-89da-3a52ddf6632c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 08:33:41.936626 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:41.950238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.sql.core [None req-9eee876e-ea68-43e3-89da-3a52ddf6632c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 08:33:41.950238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 08:33:41.950238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [parameters: {'id': '83275b15ac7441c2a7d790c6f7cbb92f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 08:33:41.950238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87945) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-9eee876e-ea68-43e3-89da-3a52ddf6632c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 83275b15ac7441c2a7d790c6f7cbb92f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 83275b15ac7441c2a7d790c6f7cbb92f. Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application conn.query(q) Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 08:33:41.957107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application result.read() Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 04 08:33:41.958089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application next(self.gen) Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application yield resource Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application next(self.gen) Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise self.value Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application session.commit() Oct 04 08:33:41.959051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self.session.flush() Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application compat.raise_( Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 08:33:41.959988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 08:33:41.962158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application conn.query(q) Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application result.read() Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 08:33:41.964496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [parameters: {'id': '83275b15ac7441c2a7d790c6f7cbb92f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:41.966912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:41.969416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:41.972051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:41.972051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 04 08:33:41.972051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 04 08:33:41.972051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:41.972051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:41.972051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 04 08:33:41.972051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 04 08:33:41.972051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 83275b15ac7441c2a7d790c6f7cbb92f. Oct 04 08:33:41.972051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:41.972051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 480/949] 10.222.0.26 () {60 vars in 1140 bytes} [Tue Oct 4 08:33:41 2022] POST /identity/v3/projects => generated 262 bytes in 179 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:41.972051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee876e-ea68-43e3-89da-3a52ddf6632c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:41.987834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d748a3-8c27-4330-87f9-8bc09b345e40 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:42.024532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b5dec-e9a2-4330-99f5-9d24a0646946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:42.025244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b5dec-e9a2-4330-99f5-9d24a0646946 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:42.025521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b5dec-e9a2-4330-99f5-9d24a0646946 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:42.025822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b5dec-e9a2-4330-99f5-9d24a0646946 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:42.026451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b5dec-e9a2-4330-99f5-9d24a0646946 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:42.026748 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b5dec-e9a2-4330-99f5-9d24a0646946 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:42.028169 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b5dec-e9a2-4330-99f5-9d24a0646946 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:42.032921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 470/950] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:42.033707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b5dec-e9a2-4330-99f5-9d24a0646946 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:42.064100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:42.077088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee876e-ea68-43e3-89da-3a52ddf6632c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:42.114056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8fb76-e99f-4e22-a664-d0d153f1bd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:42.114724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8fb76-e99f-4e22-a664-d0d153f1bd5c None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:42.115008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8fb76-e99f-4e22-a664-d0d153f1bd5c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:42.115284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8fb76-e99f-4e22-a664-d0d153f1bd5c None admin] PATH_INFO: `/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/595dac02b846405cab34dcf898ce5b5a/roles/9b17208db146464c84a15bc97da278d9` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:42.117568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8fb76-e99f-4e22-a664-d0d153f1bd5c None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3df14a62714f6bab2acdfc979b655b, user_id=595dac02b846405cab34dcf898ce5b5a, role_id=9b17208db146464c84a15bc97da278d9)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:42.119248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8fb76-e99f-4e22-a664-d0d153f1bd5c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:42.134746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 481/951] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:33:41 2022] PUT /identity/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/595dac02b846405cab34dcf898ce5b5a/roles/9b17208db146464c84a15bc97da278d9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:33:42.135544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8fb76-e99f-4e22-a664-d0d153f1bd5c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:42.158686 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b5dec-e9a2-4330-99f5-9d24a0646946 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:42.189623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:42.200952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bd493-e084-4b6f-8801-745d09831851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:42.201516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bd493-e084-4b6f-8801-745d09831851 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:42.201838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bd493-e084-4b6f-8801-745d09831851 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:42.202113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bd493-e084-4b6f-8801-745d09831851 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:42.210660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0bd493-e084-4b6f-8801-745d09831851 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:42.212230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0bd493-e084-4b6f-8801-745d09831851 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-ba0bd493-e084-4b6f-8801-745d09831851 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:42.226219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:42.227197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:42.228278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:42.228278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:42.228278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:42.228278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:42.228278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:42.228278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:42.228278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:42.228278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:42.232604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 471/952] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:42.234217 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bd493-e084-4b6f-8801-745d09831851 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:42.295959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8fb76-e99f-4e22-a664-d0d153f1bd5c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:42.346969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8ddea-d293-43b5-817c-3d6b8a67f3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:42.347743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8ddea-d293-43b5-817c-3d6b8a67f3ef None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:42.348028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8ddea-d293-43b5-817c-3d6b8a67f3ef None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:42.348374 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8ddea-d293-43b5-817c-3d6b8a67f3ef None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:42.349007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8ddea-d293-43b5-817c-3d6b8a67f3ef None admin] RBAC: Adding query filter params (type=compute) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:42.349315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8ddea-d293-43b5-817c-3d6b8a67f3ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:42.350741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8ddea-d293-43b5-817c-3d6b8a67f3ef None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:42.356399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bd493-e084-4b6f-8801-745d09831851 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:42.358538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 482/953] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:42.400296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d68267-496f-478c-803a-d2f4e18bf6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:42.401179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d68267-496f-478c-803a-d2f4e18bf6ce None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:42.401607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d68267-496f-478c-803a-d2f4e18bf6ce None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:42.402080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d68267-496f-478c-803a-d2f4e18bf6ce None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:42.402895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d68267-496f-478c-803a-d2f4e18bf6ce None admin] RBAC: Adding query filter params (name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:42.403345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d68267-496f-478c-803a-d2f4e18bf6ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:42.405037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d68267-496f-478c-803a-d2f4e18bf6ce None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:42.426194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 472/954] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:43.601921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd1b8a-cb2f-4606-8d23-2602c102de10 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:43.602468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd1b8a-cb2f-4606-8d23-2602c102de10 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:43.602468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd1b8a-cb2f-4606-8d23-2602c102de10 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:43.604348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 483/955] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:43.609882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e7608-9ab5-4c13-ac51-2dc3ed5bd38b None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:43.610273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e7608-9ab5-4c13-ac51-2dc3ed5bd38b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:43.610440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e7608-9ab5-4c13-ac51-2dc3ed5bd38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:43.629256 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-858e7608-9ab5-4c13-ac51-2dc3ed5bd38b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:43.631009 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-858e7608-9ab5-4c13-ac51-2dc3ed5bd38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:43.685677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:43.717809 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 473/956] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:44.002605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd1b8a-cb2f-4606-8d23-2602c102de10 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:44.021761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:44.123164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddd1b8a-cb2f-4606-8d23-2602c102de10 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:44.161352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466ffca-c9df-4a79-bf00-4267d261f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:44.162381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466ffca-c9df-4a79-bf00-4267d261f13f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:44.162863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466ffca-c9df-4a79-bf00-4267d261f13f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:44.163336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466ffca-c9df-4a79-bf00-4267d261f13f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:44.164130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466ffca-c9df-4a79-bf00-4267d261f13f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:44.165742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466ffca-c9df-4a79-bf00-4267d261f13f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-2466ffca-c9df-4a79-bf00-4267d261f13f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:44.174950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:44.176018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:44.176929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:44.179692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 484/957] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 08:33:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:44.185716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e7608-9ab5-4c13-ac51-2dc3ed5bd38b None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:44.294348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e7608-9ab5-4c13-ac51-2dc3ed5bd38b None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:44.334737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a23d4-ba1d-4c5a-ac0b-8f3b4d088918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:44.335357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a23d4-ba1d-4c5a-ac0b-8f3b4d088918 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:44.335639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a23d4-ba1d-4c5a-ac0b-8f3b4d088918 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:44.335904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a23d4-ba1d-4c5a-ac0b-8f3b4d088918 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:44.336539 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a23d4-ba1d-4c5a-ac0b-8f3b4d088918 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:44.336844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a23d4-ba1d-4c5a-ac0b-8f3b4d088918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:44.338258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a23d4-ba1d-4c5a-ac0b-8f3b4d088918 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:44.347887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 474/958] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 08:33:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:44.350838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466ffca-c9df-4a79-bf00-4267d261f13f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:44.433909 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:44.443352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1ec48-e353-40dc-9057-9134dabf2129 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:44.443678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1ec48-e353-40dc-9057-9134dabf2129 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:44.443929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1ec48-e353-40dc-9057-9134dabf2129 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:44.445482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 475/959] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:44.451383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f274ec1-aa48-4278-a787-c931a4d7e3a4 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:44.452013 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f274ec1-aa48-4278-a787-c931a4d7e3a4 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:44.452489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f274ec1-aa48-4278-a787-c931a4d7e3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:44.466876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466ffca-c9df-4a79-bf00-4267d261f13f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:44.482198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-1f274ec1-aa48-4278-a787-c931a4d7e3a4 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:44.483520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1f274ec1-aa48-4278-a787-c931a4d7e3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:44.506492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112df2e-9e5b-459a-b61e-65c46f485e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:44.507466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112df2e-9e5b-459a-b61e-65c46f485e00 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:44.507959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112df2e-9e5b-459a-b61e-65c46f485e00 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:44.508431 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112df2e-9e5b-459a-b61e-65c46f485e00 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:44.509280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112df2e-9e5b-459a-b61e-65c46f485e00 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:44.509838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112df2e-9e5b-459a-b61e-65c46f485e00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:44.511408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112df2e-9e5b-459a-b61e-65c46f485e00 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:44.518501 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 485/960] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 08:33:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:44.526080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112df2e-9e5b-459a-b61e-65c46f485e00 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:44.530572 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:44.579989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 476/961] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:44.579989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe7328-6038-4cc1-bc52-e7f241effb7c None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:44.579989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe7328-6038-4cc1-bc52-e7f241effb7c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:44.579989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe7328-6038-4cc1-bc52-e7f241effb7c None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:44.579989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 477/962] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:44.586259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a23bdc-ab9c-4fa3-ac6b-db2f644348b7 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:44.586615 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a23bdc-ab9c-4fa3-ac6b-db2f644348b7 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:44.586959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a23bdc-ab9c-4fa3-ac6b-db2f644348b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:44.607620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-72a23bdc-ab9c-4fa3-ac6b-db2f644348b7 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:44.609109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-72a23bdc-ab9c-4fa3-ac6b-db2f644348b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:44.647943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112df2e-9e5b-459a-b61e-65c46f485e00 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:44.691428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8c3de-2aab-4ebd-9472-0362cdbfa258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:44.692638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8c3de-2aab-4ebd-9472-0362cdbfa258 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:44.693007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8c3de-2aab-4ebd-9472-0362cdbfa258 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:44.693201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8c3de-2aab-4ebd-9472-0362cdbfa258 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:44.693882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8c3de-2aab-4ebd-9472-0362cdbfa258 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:44.695255 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8c3de-2aab-4ebd-9472-0362cdbfa258 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:44.698753 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 478/963] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:44.699786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08bfa6-d399-4439-abe6-10065e71d6f0 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:44.700070 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08bfa6-d399-4439-abe6-10065e71d6f0 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:44.700338 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08bfa6-d399-4439-abe6-10065e71d6f0 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:44.701661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 479/964] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:44.709827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cde599-3a7c-4e11-8fe8-6edf48234d09 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:44.709827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cde599-3a7c-4e11-8fe8-6edf48234d09 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:44.709827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cde599-3a7c-4e11-8fe8-6edf48234d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:44.722098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:44.727887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-97cde599-3a7c-4e11-8fe8-6edf48234d09 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:44.729395 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-97cde599-3a7c-4e11-8fe8-6edf48234d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:44.739132 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 486/965] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:33:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:33:44.818684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 480/966] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:44.863746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8c3de-2aab-4ebd-9472-0362cdbfa258 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:44.976291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cde599-3a7c-4e11-8fe8-6edf48234d09 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:44.979055 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8c3de-2aab-4ebd-9472-0362cdbfa258 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:45.013235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:45.026838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78239d55-8a52-4554-bb11-1df5173bfb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:45.027486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78239d55-8a52-4554-bb11-1df5173bfb78 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:45.027825 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78239d55-8a52-4554-bb11-1df5173bfb78 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:45.028155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78239d55-8a52-4554-bb11-1df5173bfb78 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:45.039567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78239d55-8a52-4554-bb11-1df5173bfb78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-78239d55-8a52-4554-bb11-1df5173bfb78 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:45.044387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:45.045303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:45.046166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 487/967] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Oct 4 08:33:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:45.051321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78239d55-8a52-4554-bb11-1df5173bfb78 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:45.112129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cde599-3a7c-4e11-8fe8-6edf48234d09 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:45.112513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:45.147898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f0f3c-baf1-4767-b74b-8455009de168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:45.148574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f0f3c-baf1-4767-b74b-8455009de168 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:45.148863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f0f3c-baf1-4767-b74b-8455009de168 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:45.149127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f0f3c-baf1-4767-b74b-8455009de168 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:45.150135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f0f3c-baf1-4767-b74b-8455009de168 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:45.151495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f0f3c-baf1-4767-b74b-8455009de168 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:45.158736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78239d55-8a52-4554-bb11-1df5173bfb78 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:45.165438 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 481/968] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:33:44 2022] POST /identity/v3/services => generated 242 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:33:45.166254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f0f3c-baf1-4767-b74b-8455009de168 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:45.226710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa165d-2762-468d-85d6-523e922f9c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:45.227725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa165d-2762-468d-85d6-523e922f9c7f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:45.228189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa165d-2762-468d-85d6-523e922f9c7f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:45.229768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa165d-2762-468d-85d6-523e922f9c7f None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:45.230763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa165d-2762-468d-85d6-523e922f9c7f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:45.231262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa165d-2762-468d-85d6-523e922f9c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:45.234025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa165d-2762-468d-85d6-523e922f9c7f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:45.251468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 488/969] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 08:33:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:45.252280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa165d-2762-468d-85d6-523e922f9c7f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:45.304429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:45.325606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f0f3c-baf1-4767-b74b-8455009de168 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:45.378425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e638455-8d92-40d0-9c3c-da2b96a7b936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:45.379106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e638455-8d92-40d0-9c3c-da2b96a7b936 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:45.379472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e638455-8d92-40d0-9c3c-da2b96a7b936 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:45.379857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e638455-8d92-40d0-9c3c-da2b96a7b936 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:45.380516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e638455-8d92-40d0-9c3c-da2b96a7b936 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:45.382520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e638455-8d92-40d0-9c3c-da2b96a7b936 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:45.385345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: WARNING keystone.server.flask.common [None req-4e638455-8d92-40d0-9c3c-da2b96a7b936 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 08:33:45.393539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa165d-2762-468d-85d6-523e922f9c7f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:45.404144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.sql.core [None req-4e638455-8d92-40d0-9c3c-da2b96a7b936 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 08:33:45.404144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 08:33:45.404144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [parameters: {'user_id': '980378f85c0a4e2f9397d44b48eaaa00', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 08:33:45.404144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87946) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 08:33:45.413033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-4e638455-8d92-40d0-9c3c-da2b96a7b936 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application conn.query(q) Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application result.read() Oct 04 08:33:45.422021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 08:33:45.422977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application next(self.gen) Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application yield resource Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application next(self.gen) Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise self.value Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application session.commit() Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 08:33:45.423955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self.session.flush() Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application compat.raise_( Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 08:33:45.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 08:33:45.425839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application conn.query(q) Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application result.read() Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 08:33:45.427325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [parameters: {'user_id': '980378f85c0a4e2f9397d44b48eaaa00', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:45.428191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 04 08:33:45.428993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 04 08:33:45.431846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 04 08:33:45.431846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 04 08:33:45.431846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 08:33:45.431846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 08:33:45.431846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 08:33:45.431846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:45.431846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 482/970] 10.222.0.26 () {60 vars in 1131 bytes} [Tue Oct 4 08:33:45 2022] POST /identity/v3/users => generated 160 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:45.431846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e638455-8d92-40d0-9c3c-da2b96a7b936 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:45.449638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a80ee6-5a1b-4f04-b41f-68868c89e783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:45.451007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a80ee6-5a1b-4f04-b41f-68868c89e783 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:45.451535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a80ee6-5a1b-4f04-b41f-68868c89e783 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:45.452249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a80ee6-5a1b-4f04-b41f-68868c89e783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:45.468660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a80ee6-5a1b-4f04-b41f-68868c89e783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:45.470821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a80ee6-5a1b-4f04-b41f-68868c89e783 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-64a80ee6-5a1b-4f04-b41f-68868c89e783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:45.491553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:45.492603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:45.493506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:45.493506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:45.493506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:45.495230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 489/971] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:45.500965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a80ee6-5a1b-4f04-b41f-68868c89e783 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:45.573107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e638455-8d92-40d0-9c3c-da2b96a7b936 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:45.612844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a80ee6-5a1b-4f04-b41f-68868c89e783 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:45.620133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c49ae-94bb-41d7-a742-c1747a1a60e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:45.621629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c49ae-94bb-41d7-a742-c1747a1a60e1 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:45.622157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c49ae-94bb-41d7-a742-c1747a1a60e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:45.622644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c49ae-94bb-41d7-a742-c1747a1a60e1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:45.634165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c49ae-94bb-41d7-a742-c1747a1a60e1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:45.636884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c49ae-94bb-41d7-a742-c1747a1a60e1 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:45.649367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab21915-da0c-452c-9282-e73ebf239634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:45.650079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab21915-da0c-452c-9282-e73ebf239634 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:45.650363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab21915-da0c-452c-9282-e73ebf239634 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:45.650638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab21915-da0c-452c-9282-e73ebf239634 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:45.651285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab21915-da0c-452c-9282-e73ebf239634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:45.651593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab21915-da0c-452c-9282-e73ebf239634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:45.653028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab21915-da0c-452c-9282-e73ebf239634 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-172c49ae-94bb-41d7-a742-c1747a1a60e1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:45.654065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:45.655019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:45.655983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:45.655983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:45.655983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:45.655983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:45.655983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:45.655983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:45.655983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:45.655983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:45.655983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 08:33:45.655983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:45.659021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 483/972] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:33:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:45.666945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c49ae-94bb-41d7-a742-c1747a1a60e1 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:45.677574 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 490/973] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:45.678736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab21915-da0c-452c-9282-e73ebf239634 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:45.680160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:45.718754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:45.799428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab21915-da0c-452c-9282-e73ebf239634 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:45.806092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c49ae-94bb-41d7-a742-c1747a1a60e1 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:45.836893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650c9e2-6c10-4e08-8b9d-087a4f58d617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:45.837917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650c9e2-6c10-4e08-8b9d-087a4f58d617 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:45.838430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650c9e2-6c10-4e08-8b9d-087a4f58d617 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:45.838901 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650c9e2-6c10-4e08-8b9d-087a4f58d617 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:45.846242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650c9e2-6c10-4e08-8b9d-087a4f58d617 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:45.848667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650c9e2-6c10-4e08-8b9d-087a4f58d617 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:45.849663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdc2f8-5b8f-4b30-b553-101af74d48e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:45.850297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdc2f8-5b8f-4b30-b553-101af74d48e9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:45.850578 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdc2f8-5b8f-4b30-b553-101af74d48e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:45.850855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdc2f8-5b8f-4b30-b553-101af74d48e9 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:45.851810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacdc2f8-5b8f-4b30-b553-101af74d48e9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:45.852114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacdc2f8-5b8f-4b30-b553-101af74d48e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:45.854406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacdc2f8-5b8f-4b30-b553-101af74d48e9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-9650c9e2-6c10-4e08-8b9d-087a4f58d617 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:45.862438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:45.863381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:45.864405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:45.864405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:45.864405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:45.864405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:45.864405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:45.864405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:45.864405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:45.864405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:45.864405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 08:33:45.864405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:45.866929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 491/974] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 08:33:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:45.872339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650c9e2-6c10-4e08-8b9d-087a4f58d617 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:45.909798 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 484/975] 10.222.0.26 () {56 vars in 1101 bytes} [Tue Oct 4 08:33:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:45.988471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:46.004776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650c9e2-6c10-4e08-8b9d-087a4f58d617 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:46.044041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aedcd8-d09d-42e3-9289-ecb237f62215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:46.044917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aedcd8-d09d-42e3-9289-ecb237f62215 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:46.045376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aedcd8-d09d-42e3-9289-ecb237f62215 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:46.045832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aedcd8-d09d-42e3-9289-ecb237f62215 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:46.046753 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aedcd8-d09d-42e3-9289-ecb237f62215 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:46.047223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aedcd8-d09d-42e3-9289-ecb237f62215 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:46.048941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aedcd8-d09d-42e3-9289-ecb237f62215 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:46.070625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 492/976] 10.222.0.26 () {56 vars in 1149 bytes} [Tue Oct 4 08:33:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:46.076623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdc2f8-5b8f-4b30-b553-101af74d48e9 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:46.148142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:46.186160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdc2f8-5b8f-4b30-b553-101af74d48e9 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:46.221354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e508fcb-0060-450e-a23f-e75cd9790086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:46.222090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e508fcb-0060-450e-a23f-e75cd9790086 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:46.222445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e508fcb-0060-450e-a23f-e75cd9790086 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:46.222793 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e508fcb-0060-450e-a23f-e75cd9790086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:46.290517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e508fcb-0060-450e-a23f-e75cd9790086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:46.292049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e508fcb-0060-450e-a23f-e75cd9790086 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:46.353313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:46.389084 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 485/977] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:46 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:33:46.394619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aedcd8-d09d-42e3-9289-ecb237f62215 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:46.494650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aedcd8-d09d-42e3-9289-ecb237f62215 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:46.529510 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85914c2c-e41b-43e3-bc79-eda3fad55321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:46.530170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85914c2c-e41b-43e3-bc79-eda3fad55321 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:46.530452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85914c2c-e41b-43e3-bc79-eda3fad55321 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:46.530723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85914c2c-e41b-43e3-bc79-eda3fad55321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:46.541484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85914c2c-e41b-43e3-bc79-eda3fad55321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:46.542989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85914c2c-e41b-43e3-bc79-eda3fad55321 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:46.549895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-85914c2c-e41b-43e3-bc79-eda3fad55321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:46.559565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:46.560696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:46.562026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:46.562026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:46.562026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:46.565830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 493/978] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:46.567532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e508fcb-0060-450e-a23f-e75cd9790086 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:46.673015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e508fcb-0060-450e-a23f-e75cd9790086 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:46.706833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c27c6-dbdc-42c7-899c-7b512bfc7694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:46.707953 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c27c6-dbdc-42c7-899c-7b512bfc7694 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:46.708229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c27c6-dbdc-42c7-899c-7b512bfc7694 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:46.708495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c27c6-dbdc-42c7-899c-7b512bfc7694 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:46.709133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c27c6-dbdc-42c7-899c-7b512bfc7694 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:46.709433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c27c6-dbdc-42c7-899c-7b512bfc7694 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:46.710812 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c27c6-dbdc-42c7-899c-7b512bfc7694 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:46.722252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 486/979] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:46.727674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85914c2c-e41b-43e3-bc79-eda3fad55321 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:46.828101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85914c2c-e41b-43e3-bc79-eda3fad55321 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:46.861145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397832c0-850d-48a5-8fa2-2ebd21b31ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:46.861791 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397832c0-850d-48a5-8fa2-2ebd21b31ac2 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:46.862071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397832c0-850d-48a5-8fa2-2ebd21b31ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:46.862343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397832c0-850d-48a5-8fa2-2ebd21b31ac2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:46.869451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397832c0-850d-48a5-8fa2-2ebd21b31ac2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:46.870925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397832c0-850d-48a5-8fa2-2ebd21b31ac2 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-397832c0-850d-48a5-8fa2-2ebd21b31ac2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:46.881952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:46.882850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:46.883756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:46.883756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:46.883756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:46.883756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:46.883756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:46.883756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:46.883756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:46.883756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:46.884759 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 494/980] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:46.888739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c27c6-dbdc-42c7-899c-7b512bfc7694 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:46.918327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:47.002926 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c27c6-dbdc-42c7-899c-7b512bfc7694 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:47.040122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ecc63-e81b-4a10-aa6e-f063d94ce742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:47.040873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ecc63-e81b-4a10-aa6e-f063d94ce742 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:47.041253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ecc63-e81b-4a10-aa6e-f063d94ce742 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:47.041622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ecc63-e81b-4a10-aa6e-f063d94ce742 None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:47.042429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ecc63-e81b-4a10-aa6e-f063d94ce742 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:47.042851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ecc63-e81b-4a10-aa6e-f063d94ce742 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:47.044435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ecc63-e81b-4a10-aa6e-f063d94ce742 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:47.056993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 487/981] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:47.062750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397832c0-850d-48a5-8fa2-2ebd21b31ac2 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:47.080901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ee1fc-e3e7-4e0f-a13e-2ce0fdaa2d6f None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:47.081172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ee1fc-e3e7-4e0f-a13e-2ce0fdaa2d6f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:47.081454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ee1fc-e3e7-4e0f-a13e-2ce0fdaa2d6f None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:47.083696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 488/982] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:33:47.088577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79842ec2-9651-41bb-add8-3e36c99317e6 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:47.088863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79842ec2-9651-41bb-add8-3e36c99317e6 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:47.089143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79842ec2-9651-41bb-add8-3e36c99317e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:47.108476 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-79842ec2-9651-41bb-add8-3e36c99317e6 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:47.110021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-79842ec2-9651-41bb-add8-3e36c99317e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:47.128810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:47.181907 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397832c0-850d-48a5-8fa2-2ebd21b31ac2 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:47.185750 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 489/983] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:47.217541 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6065cb-2b62-459c-9d03-fd85c2bdea8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:47.218176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6065cb-2b62-459c-9d03-fd85c2bdea8a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:47.218462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6065cb-2b62-459c-9d03-fd85c2bdea8a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:47.218737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6065cb-2b62-459c-9d03-fd85c2bdea8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:47.219450 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6065cb-2b62-459c-9d03-fd85c2bdea8a None admin] RBAC: Adding query filter params (role.id=9b17208db146464c84a15bc97da278d9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=595dac02b846405cab34dcf898ce5b5a) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:47.219779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6065cb-2b62-459c-9d03-fd85c2bdea8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:47.221178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6065cb-2b62-459c-9d03-fd85c2bdea8a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:47.227358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 495/984] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:47 2022] GET /identity/v3/role_assignments?user.id=595dac02b846405cab34dcf898ce5b5a&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=9b17208db146464c84a15bc97da278d9 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:47.418149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044104a-09d5-49c4-970e-ade8b6e227e7 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:47.418358 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044104a-09d5-49c4-970e-ade8b6e227e7 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:47.418630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044104a-09d5-49c4-970e-ade8b6e227e7 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:47.419965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 490/985] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:47.425185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632f1fc-b942-4668-a61e-e4de1c97b7bf None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:47.425474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632f1fc-b942-4668-a61e-e4de1c97b7bf None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:47.425788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632f1fc-b942-4668-a61e-e4de1c97b7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:47.445270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6632f1fc-b942-4668-a61e-e4de1c97b7bf None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:47.446830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6632f1fc-b942-4668-a61e-e4de1c97b7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:47.493146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044104a-09d5-49c4-970e-ade8b6e227e7 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:47.511737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:47.521732 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 496/986] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:47.602450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044104a-09d5-49c4-970e-ade8b6e227e7 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:47.637236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b9437-9750-4652-bbd3-95921a43ae35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:47.637980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b9437-9750-4652-bbd3-95921a43ae35 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:47.638328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b9437-9750-4652-bbd3-95921a43ae35 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:47.638671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b9437-9750-4652-bbd3-95921a43ae35 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:47.639320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b9437-9750-4652-bbd3-95921a43ae35 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:47.640743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b9437-9750-4652-bbd3-95921a43ae35 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-616b9437-9750-4652-bbd3-95921a43ae35 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:47.649938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:47.651229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:47.652110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:47.656520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 491/987] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 08:33:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:47.658032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632f1fc-b942-4668-a61e-e4de1c97b7bf None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:47.718005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:47.770449 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632f1fc-b942-4668-a61e-e4de1c97b7bf None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:47.784126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b9437-9750-4652-bbd3-95921a43ae35 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:47.806108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdb943-cf95-47a5-af48-0b710f859d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:47.806719 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdb943-cf95-47a5-af48-0b710f859d10 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:47.806999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdb943-cf95-47a5-af48-0b710f859d10 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:47.807269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdb943-cf95-47a5-af48-0b710f859d10 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:47.807900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdb943-cf95-47a5-af48-0b710f859d10 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:47.808202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdb943-cf95-47a5-af48-0b710f859d10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:47.809574 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdb943-cf95-47a5-af48-0b710f859d10 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:47.814613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 497/988] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 08:33:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:47.820142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdb943-cf95-47a5-af48-0b710f859d10 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:47.870705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:47.892759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b9437-9750-4652-bbd3-95921a43ae35 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:47.923803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdb943-cf95-47a5-af48-0b710f859d10 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:47.928797 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2177c60-3a03-40f5-b303-6c952ba1897f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:47.929628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177c60-3a03-40f5-b303-6c952ba1897f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:47.930083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177c60-3a03-40f5-b303-6c952ba1897f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:47.930516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2177c60-3a03-40f5-b303-6c952ba1897f None admin] PATH_INFO: `/v3/services/compute` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:47.931322 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2177c60-3a03-40f5-b303-6c952ba1897f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:47.932932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2177c60-3a03-40f5-b303-6c952ba1897f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:47.939635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-b2177c60-3a03-40f5-b303-6c952ba1897f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:47.944602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:47.945828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:33:47.946685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:47.947445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 492/989] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:47.952550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2177c60-3a03-40f5-b303-6c952ba1897f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:47.961957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf956a8-52e0-49aa-b066-4e3f6fc4b4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:47.962651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf956a8-52e0-49aa-b066-4e3f6fc4b4c7 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:47.962945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf956a8-52e0-49aa-b066-4e3f6fc4b4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:47.963204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf956a8-52e0-49aa-b066-4e3f6fc4b4c7 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:47.963840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf956a8-52e0-49aa-b066-4e3f6fc4b4c7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:47.964136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf956a8-52e0-49aa-b066-4e3f6fc4b4c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:47.965499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf956a8-52e0-49aa-b066-4e3f6fc4b4c7 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:47.975738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 498/990] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 08:33:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:47.976692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf956a8-52e0-49aa-b066-4e3f6fc4b4c7 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:48.071777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2177c60-3a03-40f5-b303-6c952ba1897f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:48.083202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf956a8-52e0-49aa-b066-4e3f6fc4b4c7 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:48.108485 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47349548-9481-41a8-9a36-c6180dfcf77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:48.109090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47349548-9481-41a8-9a36-c6180dfcf77d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:48.109375 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47349548-9481-41a8-9a36-c6180dfcf77d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:48.109632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47349548-9481-41a8-9a36-c6180dfcf77d None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:48.110280 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47349548-9481-41a8-9a36-c6180dfcf77d None admin] RBAC: Adding query filter params (name=compute) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:48.110579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47349548-9481-41a8-9a36-c6180dfcf77d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:48.111934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47349548-9481-41a8-9a36-c6180dfcf77d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:48.121314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b97a7-08bd-4ae2-823b-ef0814c93c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:48.121510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 493/991] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:48.121605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b97a7-08bd-4ae2-823b-ef0814c93c67 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:48.121605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b97a7-08bd-4ae2-823b-ef0814c93c67 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:48.121605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b97a7-08bd-4ae2-823b-ef0814c93c67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:48.121605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b97a7-08bd-4ae2-823b-ef0814c93c67 None admin] RBAC: Adding query filter params (interface=admin, service_id=94dfb87bd7204b0798c8c0d6e717b4e5, region_id=RegionOne) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:48.121605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b97a7-08bd-4ae2-823b-ef0814c93c67 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:48.121605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b97a7-08bd-4ae2-823b-ef0814c93c67 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:48.122373 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47349548-9481-41a8-9a36-c6180dfcf77d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:48.126523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 499/992] 10.222.0.26 () {56 vars in 1249 bytes} [Tue Oct 4 08:33:47 2022] GET /identity/v3/endpoints?service_id=94dfb87bd7204b0798c8c0d6e717b4e5&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:48.132942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b97a7-08bd-4ae2-823b-ef0814c93c67 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:48.140337 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:48.179555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:48.233903 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b97a7-08bd-4ae2-823b-ef0814c93c67 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:48.234350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47349548-9481-41a8-9a36-c6180dfcf77d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:48.269650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38156ca6-e998-4864-9fb6-bb276566f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:48.270722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c600c81-c21b-43b2-b3c2-bc0455d2268c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:48.270859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38156ca6-e998-4864-9fb6-bb276566f183 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:48.271124 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38156ca6-e998-4864-9fb6-bb276566f183 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:48.271330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c600c81-c21b-43b2-b3c2-bc0455d2268c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:48.271432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38156ca6-e998-4864-9fb6-bb276566f183 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:48.271611 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c600c81-c21b-43b2-b3c2-bc0455d2268c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:48.271882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c600c81-c21b-43b2-b3c2-bc0455d2268c None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:48.271979 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38156ca6-e998-4864-9fb6-bb276566f183 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:48.272282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38156ca6-e998-4864-9fb6-bb276566f183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:48.272484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c600c81-c21b-43b2-b3c2-bc0455d2268c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:48.273632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38156ca6-e998-4864-9fb6-bb276566f183 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:48.273868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c600c81-c21b-43b2-b3c2-bc0455d2268c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:48.278528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 494/993] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:48.287275 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 500/994] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 08:33:48 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:33:48.287478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38156ca6-e998-4864-9fb6-bb276566f183 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:48.367037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:48.400799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38156ca6-e998-4864-9fb6-bb276566f183 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:48.436736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078cb74-802b-41bf-b0aa-8474faca6c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:48.437512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078cb74-802b-41bf-b0aa-8474faca6c30 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:48.437913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078cb74-802b-41bf-b0aa-8474faca6c30 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:48.438279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078cb74-802b-41bf-b0aa-8474faca6c30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:48.439093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078cb74-802b-41bf-b0aa-8474faca6c30 None admin] RBAC: Adding query filter params (interface=public, service_id=8ebad24003154836a9a65747db896393, region_id=RegionOne) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:48.439518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078cb74-802b-41bf-b0aa-8474faca6c30 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:48.441005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078cb74-802b-41bf-b0aa-8474faca6c30 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:48.451511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 495/995] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 08:33:48 2022] GET /identity/v3/endpoints?service_id=8ebad24003154836a9a65747db896393&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:48.452819 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c600c81-c21b-43b2-b3c2-bc0455d2268c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:48.563670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c600c81-c21b-43b2-b3c2-bc0455d2268c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:48.573247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:48.599633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d699b5-45b3-4969-b00d-f402679d9d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:48.600329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d699b5-45b3-4969-b00d-f402679d9d98 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:48.600686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d699b5-45b3-4969-b00d-f402679d9d98 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:48.601035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d699b5-45b3-4969-b00d-f402679d9d98 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:48.601731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d699b5-45b3-4969-b00d-f402679d9d98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:48.603163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d699b5-45b3-4969-b00d-f402679d9d98 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:48.613928 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 501/996] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 08:33:48 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:33:49.417574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3636294-60fa-44ff-8807-91366cef4309 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:49.418194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3636294-60fa-44ff-8807-91366cef4309 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:49.418194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3636294-60fa-44ff-8807-91366cef4309 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:49.419637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 496/997] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:49.424624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fc97f-35ea-439f-a2f1-d8287066a057 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:49.425015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fc97f-35ea-439f-a2f1-d8287066a057 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:49.425211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350fc97f-35ea-439f-a2f1-d8287066a057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:49.443530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-350fc97f-35ea-439f-a2f1-d8287066a057 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:49.444700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-350fc97f-35ea-439f-a2f1-d8287066a057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:49.515490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 502/998] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:49.778756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3636294-60fa-44ff-8807-91366cef4309 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:49.877484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3636294-60fa-44ff-8807-91366cef4309 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:49.909469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe9e0c-c482-46a1-8e97-a91e0745827c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:49.910111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe9e0c-c482-46a1-8e97-a91e0745827c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:49.910947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe9e0c-c482-46a1-8e97-a91e0745827c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:49.911222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe9e0c-c482-46a1-8e97-a91e0745827c None admin] PATH_INFO: `/v3/services/image` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:49.911791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe9e0c-c482-46a1-8e97-a91e0745827c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:49.913139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe9e0c-c482-46a1-8e97-a91e0745827c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:49.918619 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-1bfe9e0c-c482-46a1-8e97-a91e0745827c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:49.926068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:49.927284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:33:49.928332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:49.930965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 497/999] 10.222.0.26 () {56 vars in 1105 bytes} [Tue Oct 4 08:33:49 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:49.936541 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fc97f-35ea-439f-a2f1-d8287066a057 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:50.037059 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350fc97f-35ea-439f-a2f1-d8287066a057 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:50.038486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:50.072841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4fc96-be25-4f4d-a3a0-4aa0047218ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:50.073448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4fc96-be25-4f4d-a3a0-4aa0047218ca None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:50.073754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4fc96-be25-4f4d-a3a0-4aa0047218ca None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:50.074031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4fc96-be25-4f4d-a3a0-4aa0047218ca None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:50.074657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4fc96-be25-4f4d-a3a0-4aa0047218ca None admin] RBAC: Adding query filter params (name=image) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:50.074964 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4fc96-be25-4f4d-a3a0-4aa0047218ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:50.076334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4fc96-be25-4f4d-a3a0-4aa0047218ca None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:50.081335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 503/1000] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:50.087471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe9e0c-c482-46a1-8e97-a91e0745827c None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:50.186294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe9e0c-c482-46a1-8e97-a91e0745827c None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:50.219782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5905e2-dcc7-46d6-9bdf-42fbefd13bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:50.220383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5905e2-dcc7-46d6-9bdf-42fbefd13bb4 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:50.220662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5905e2-dcc7-46d6-9bdf-42fbefd13bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:50.220940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5905e2-dcc7-46d6-9bdf-42fbefd13bb4 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:50.221562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5905e2-dcc7-46d6-9bdf-42fbefd13bb4 None admin] RBAC: Adding query filter params (type=image) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:50.221895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5905e2-dcc7-46d6-9bdf-42fbefd13bb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:50.223834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5905e2-dcc7-46d6-9bdf-42fbefd13bb4 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:50.228406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 498/1001] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:50.460626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c362a-21b3-413e-9b17-556292edec81 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:50.461127 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c362a-21b3-413e-9b17-556292edec81 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:50.461127 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c362a-21b3-413e-9b17-556292edec81 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:50.462492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 504/1002] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:50.467614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bfcaa-41bc-4c9a-8f5a-530b09ac73cb None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:50.468001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bfcaa-41bc-4c9a-8f5a-530b09ac73cb None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:50.468185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bfcaa-41bc-4c9a-8f5a-530b09ac73cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:50.486520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-007bfcaa-41bc-4c9a-8f5a-530b09ac73cb None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:50.487708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-007bfcaa-41bc-4c9a-8f5a-530b09ac73cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:50.545969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:50.555902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 499/1003] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:50.755107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a9db5-9145-4a58-95e7-279731ae2532 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:50.755303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a9db5-9145-4a58-95e7-279731ae2532 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:50.755579 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a9db5-9145-4a58-95e7-279731ae2532 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:50.757039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 505/1004] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:50.761962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dfa52-3430-4380-934a-e263dbcd05cd None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:50.762249 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dfa52-3430-4380-934a-e263dbcd05cd None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:50.762517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dfa52-3430-4380-934a-e263dbcd05cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:50.780801 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-501dfa52-3430-4380-934a-e263dbcd05cd None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:50.781989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-501dfa52-3430-4380-934a-e263dbcd05cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:50.820845 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a9db5-9145-4a58-95e7-279731ae2532 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:50.848586 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 500/1005] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:50.921661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a9db5-9145-4a58-95e7-279731ae2532 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:50.955738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9cdf8-5033-42d6-b8b9-867c62f9c7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:50.956368 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9cdf8-5033-42d6-b8b9-867c62f9c7de None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:50.956644 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9cdf8-5033-42d6-b8b9-867c62f9c7de None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:50.956915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9cdf8-5033-42d6-b8b9-867c62f9c7de None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:50.957542 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9cdf8-5033-42d6-b8b9-867c62f9c7de None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:50.958939 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9cdf8-5033-42d6-b8b9-867c62f9c7de None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-bbd9cdf8-5033-42d6-b8b9-867c62f9c7de None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:50.967529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:50.968522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 08:33:50.969373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:50.973997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 506/1006] 10.222.0.26 () {56 vars in 1126 bytes} [Tue Oct 4 08:33:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:50.975087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501dfa52-3430-4380-934a-e263dbcd05cd None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:51.071391 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501dfa52-3430-4380-934a-e263dbcd05cd None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:51.094671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:51.108136 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be45b5-9c3b-44c1-a28c-bdb67d81ca37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:51.108758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be45b5-9c3b-44c1-a28c-bdb67d81ca37 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:51.109039 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be45b5-9c3b-44c1-a28c-bdb67d81ca37 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:51.109314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be45b5-9c3b-44c1-a28c-bdb67d81ca37 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:51.109969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be45b5-9c3b-44c1-a28c-bdb67d81ca37 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:51.110452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be45b5-9c3b-44c1-a28c-bdb67d81ca37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:51.111411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9cdf8-5033-42d6-b8b9-867c62f9c7de None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:51.111643 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be45b5-9c3b-44c1-a28c-bdb67d81ca37 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:51.121792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 501/1007] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 08:33:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:51.124713 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be45b5-9c3b-44c1-a28c-bdb67d81ca37 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:51.146300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:51.218085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9cdf8-5033-42d6-b8b9-867c62f9c7de None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:51.229838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be45b5-9c3b-44c1-a28c-bdb67d81ca37 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:51.255420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68816bc-8526-46dc-8577-f81d9ac68ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:51.256247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68816bc-8526-46dc-8577-f81d9ac68ed1 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:51.256697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68816bc-8526-46dc-8577-f81d9ac68ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:51.257110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68816bc-8526-46dc-8577-f81d9ac68ed1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:51.257888 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68816bc-8526-46dc-8577-f81d9ac68ed1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:51.259386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68816bc-8526-46dc-8577-f81d9ac68ed1 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:51.266000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f4840-acf2-41d9-bb8f-6f2e8462aa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:51.266595 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f4840-acf2-41d9-bb8f-6f2e8462aa53 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:51.266872 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f4840-acf2-41d9-bb8f-6f2e8462aa53 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:51.267143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f4840-acf2-41d9-bb8f-6f2e8462aa53 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:51.267758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f4840-acf2-41d9-bb8f-6f2e8462aa53 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:51.268060 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f4840-acf2-41d9-bb8f-6f2e8462aa53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-d68816bc-8526-46dc-8577-f81d9ac68ed1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:51.269284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:51.270962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 08:33:51.271986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:51.272865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f4840-acf2-41d9-bb8f-6f2e8462aa53 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:51.274146 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 507/1008] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:51.277958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 502/1009] 10.222.0.26 () {56 vars in 1122 bytes} [Tue Oct 4 08:33:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:51.281275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f4840-acf2-41d9-bb8f-6f2e8462aa53 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:51.281393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68816bc-8526-46dc-8577-f81d9ac68ed1 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:51.336229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:51.380451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:51.387419 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68816bc-8526-46dc-8577-f81d9ac68ed1 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:51.388546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f4840-acf2-41d9-bb8f-6f2e8462aa53 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:51.422830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cb8b3-fb01-4228-a607-f63b8501f3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:51.423345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cb8b3-fb01-4228-a607-f63b8501f3e9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:51.423632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cb8b3-fb01-4228-a607-f63b8501f3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:51.423906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cb8b3-fb01-4228-a607-f63b8501f3e9 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:51.424535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cb8b3-fb01-4228-a607-f63b8501f3e9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:51.424648 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a744c1-62d8-4bd4-b06b-0e43143b5d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:51.424841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cb8b3-fb01-4228-a607-f63b8501f3e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:51.425565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a744c1-62d8-4bd4-b06b-0e43143b5d5d None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:51.426631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a744c1-62d8-4bd4-b06b-0e43143b5d5d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:51.426783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cb8b3-fb01-4228-a607-f63b8501f3e9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:51.427122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a744c1-62d8-4bd4-b06b-0e43143b5d5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:51.428369 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a744c1-62d8-4bd4-b06b-0e43143b5d5d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:51.430949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a744c1-62d8-4bd4-b06b-0e43143b5d5d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:51.433126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 508/1010] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:51.438649 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cb8b3-fb01-4228-a607-f63b8501f3e9 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:51.457086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 503/1011] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:33:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:33:51.553960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cb8b3-fb01-4228-a607-f63b8501f3e9 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:51.600948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edd8a8-9870-4078-bb5e-dbac1dde9766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:51.601482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edd8a8-9870-4078-bb5e-dbac1dde9766 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:51.601792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edd8a8-9870-4078-bb5e-dbac1dde9766 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:51.602061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edd8a8-9870-4078-bb5e-dbac1dde9766 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:51.602690 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edd8a8-9870-4078-bb5e-dbac1dde9766 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:51.602991 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edd8a8-9870-4078-bb5e-dbac1dde9766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:51.604655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edd8a8-9870-4078-bb5e-dbac1dde9766 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:51.615099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 509/1012] 10.222.0.26 () {56 vars in 1112 bytes} [Tue Oct 4 08:33:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:51.616121 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a744c1-62d8-4bd4-b06b-0e43143b5d5d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:51.723717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a744c1-62d8-4bd4-b06b-0e43143b5d5d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:51.736646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:51.760757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f3542-3b0b-40bc-b15c-fa250a8e3289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:51.761363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f3542-3b0b-40bc-b15c-fa250a8e3289 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:51.761642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f3542-3b0b-40bc-b15c-fa250a8e3289 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:51.761940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f3542-3b0b-40bc-b15c-fa250a8e3289 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:51.762610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f3542-3b0b-40bc-b15c-fa250a8e3289 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:51.763984 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f3542-3b0b-40bc-b15c-fa250a8e3289 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:51.783666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 504/1013] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:33:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:33:52.402113 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564fbee-3051-437c-ab6b-e3ce7e2c1c17 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:52.402503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564fbee-3051-437c-ab6b-e3ce7e2c1c17 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:52.402679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564fbee-3051-437c-ab6b-e3ce7e2c1c17 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:52.404056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 510/1014] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:52.409254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a04b98-1da2-4d3f-9091-8751d7c14640 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:52.409545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a04b98-1da2-4d3f-9091-8751d7c14640 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:52.409850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a04b98-1da2-4d3f-9091-8751d7c14640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:52.428462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a1a04b98-1da2-4d3f-9091-8751d7c14640 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:52.429643 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a1a04b98-1da2-4d3f-9091-8751d7c14640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:52.501576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 505/1015] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:52 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:52.775407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564fbee-3051-437c-ab6b-e3ce7e2c1c17 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:52.801653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:52.888005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564fbee-3051-437c-ab6b-e3ce7e2c1c17 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:52.921922 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c5d6a-8ddb-486f-bd1e-b015add15b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:52.922699 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c5d6a-8ddb-486f-bd1e-b015add15b4a None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:52.923126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c5d6a-8ddb-486f-bd1e-b015add15b4a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:52.923544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c5d6a-8ddb-486f-bd1e-b015add15b4a None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:52.924344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c5d6a-8ddb-486f-bd1e-b015add15b4a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:52.926421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c5d6a-8ddb-486f-bd1e-b015add15b4a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:52.943447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 511/1016] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:33:52 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:33:53.633383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4017d3-a8b7-4fb8-b80d-224cb18d0559 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:53.633931 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4017d3-a8b7-4fb8-b80d-224cb18d0559 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:53.634009 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4017d3-a8b7-4fb8-b80d-224cb18d0559 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:53.635319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 506/1017] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:53.640478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbe6b7-cf2c-4c3e-9254-123a374762f0 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:53.640768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbe6b7-cf2c-4c3e-9254-123a374762f0 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:53.641054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbe6b7-cf2c-4c3e-9254-123a374762f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:53.660951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-f1fbe6b7-cf2c-4c3e-9254-123a374762f0 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:53.662350 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f1fbe6b7-cf2c-4c3e-9254-123a374762f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:53.738133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 512/1018] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:53.989988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81d1c4-4baf-4968-bc83-017df007bbe1 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:53.990310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81d1c4-4baf-4968-bc83-017df007bbe1 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:53.990481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81d1c4-4baf-4968-bc83-017df007bbe1 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:53.991924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 507/1019] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:33:53.996843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fff71-dbac-446f-8f76-39fd2c83458a None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:53.997309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fff71-dbac-446f-8f76-39fd2c83458a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:53.997398 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fff71-dbac-446f-8f76-39fd2c83458a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:54.005263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81d1c4-4baf-4968-bc83-017df007bbe1 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:54.005733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:54.017505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-eb9fff71-dbac-446f-8f76-39fd2c83458a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:54.018723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-eb9fff71-dbac-446f-8f76-39fd2c83458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:54.084271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 513/1020] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:54.104099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:54.113939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81d1c4-4baf-4968-bc83-017df007bbe1 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:54.147893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb89b9-2d17-4e46-80af-26452152bc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:54.148440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb89b9-2d17-4e46-80af-26452152bc94 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:54.148717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb89b9-2d17-4e46-80af-26452152bc94 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:54.148991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb89b9-2d17-4e46-80af-26452152bc94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:54.150171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb89b9-2d17-4e46-80af-26452152bc94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:54.151597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eb89b9-2d17-4e46-80af-26452152bc94 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:54.153296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 508/1021] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:54.158760 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fff71-dbac-446f-8f76-39fd2c83458a None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:54.258752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fff71-dbac-446f-8f76-39fd2c83458a None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:54.291633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cce8a1-4aad-4295-9540-24244b4e2fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:54.292236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cce8a1-4aad-4295-9540-24244b4e2fd4 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:54.292798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cce8a1-4aad-4295-9540-24244b4e2fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:54.293068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cce8a1-4aad-4295-9540-24244b4e2fd4 None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:54.293722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cce8a1-4aad-4295-9540-24244b4e2fd4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:54.295142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cce8a1-4aad-4295-9540-24244b4e2fd4 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:54.299771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:54.326488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 514/1022] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:33:54 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:33:54.359220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb89b9-2d17-4e46-80af-26452152bc94 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:54.468614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eb89b9-2d17-4e46-80af-26452152bc94 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:54.511037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f76649-d580-4298-8a67-a0351ed88bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:54.511589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f76649-d580-4298-8a67-a0351ed88bcc None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:54.511870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f76649-d580-4298-8a67-a0351ed88bcc None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:54.512142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f76649-d580-4298-8a67-a0351ed88bcc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:54.522316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f76649-d580-4298-8a67-a0351ed88bcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-60f76649-d580-4298-8a67-a0351ed88bcc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:54.527005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:54.527939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:54.529361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 509/1023] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Oct 4 08:33:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:54.533968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cce8a1-4aad-4295-9540-24244b4e2fd4 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:54.639137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cce8a1-4aad-4295-9540-24244b4e2fd4 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:54.681624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536d673-acb5-4541-98c6-0be0a4d16529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:54.682278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536d673-acb5-4541-98c6-0be0a4d16529 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:54.682560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536d673-acb5-4541-98c6-0be0a4d16529 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:54.682835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536d673-acb5-4541-98c6-0be0a4d16529 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:54.683500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536d673-acb5-4541-98c6-0be0a4d16529 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:54.683807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536d673-acb5-4541-98c6-0be0a4d16529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:54.685170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536d673-acb5-4541-98c6-0be0a4d16529 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:54.698517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 515/1024] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 08:33:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:54.699885 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f76649-d580-4298-8a67-a0351ed88bcc None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:54.725954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:54.805101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f76649-d580-4298-8a67-a0351ed88bcc None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:54.838516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370a50f-597a-4aa2-bc7c-7900da651789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:54.839139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370a50f-597a-4aa2-bc7c-7900da651789 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:54.839413 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370a50f-597a-4aa2-bc7c-7900da651789 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:54.839679 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370a50f-597a-4aa2-bc7c-7900da651789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:54.851922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370a50f-597a-4aa2-bc7c-7900da651789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:54.851922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1370a50f-597a-4aa2-bc7c-7900da651789 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-1370a50f-597a-4aa2-bc7c-7900da651789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:54.867139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:54.868113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:54.869084 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:54.869084 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:54.869084 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:54.872985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 510/1025] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:54.873869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536d673-acb5-4541-98c6-0be0a4d16529 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:54.922548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:54.976230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536d673-acb5-4541-98c6-0be0a4d16529 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:55.013364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598b7fa-85be-4f29-a150-a06c405c4476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:55.014029 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598b7fa-85be-4f29-a150-a06c405c4476 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:55.014285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598b7fa-85be-4f29-a150-a06c405c4476 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:55.014537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598b7fa-85be-4f29-a150-a06c405c4476 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:55.015184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598b7fa-85be-4f29-a150-a06c405c4476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:55.015484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598b7fa-85be-4f29-a150-a06c405c4476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:55.016848 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598b7fa-85be-4f29-a150-a06c405c4476 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:55.029587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 516/1026] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:55.034898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370a50f-597a-4aa2-bc7c-7900da651789 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:55.103294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b12c7-9170-4932-b9f6-b4c24a46ddf0 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:55.103737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b12c7-9170-4932-b9f6-b4c24a46ddf0 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:55.104023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b12c7-9170-4932-b9f6-b4c24a46ddf0 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:55.105570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 517/1027] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:33:55.110267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949e95c-a74d-49be-826f-4a4454309167 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:55.110549 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949e95c-a74d-49be-826f-4a4454309167 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:55.110821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949e95c-a74d-49be-826f-4a4454309167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:55.112071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:55.129656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-9949e95c-a74d-49be-826f-4a4454309167 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:55.130862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9949e95c-a74d-49be-826f-4a4454309167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:55.138943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370a50f-597a-4aa2-bc7c-7900da651789 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:55.176475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc83930-85c8-4eb9-8de6-77da0c722db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:55.177411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc83930-85c8-4eb9-8de6-77da0c722db3 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:55.177864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc83930-85c8-4eb9-8de6-77da0c722db3 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:55.178149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc83930-85c8-4eb9-8de6-77da0c722db3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:55.186077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc83930-85c8-4eb9-8de6-77da0c722db3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:55.187511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc83930-85c8-4eb9-8de6-77da0c722db3 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-0fc83930-85c8-4eb9-8de6-77da0c722db3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:55.198569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:55.199544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:55.200535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:55.200535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:55.200535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:55.200535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:55.200535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:55.200535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:55.200535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:55.200535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:55.200535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:33:55.200535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:55.200535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 511/1028] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 08:33:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:55.203304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 518/1029] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:55.205205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc83930-85c8-4eb9-8de6-77da0c722db3 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:55.307271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc83930-85c8-4eb9-8de6-77da0c722db3 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:55.340220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e9e0c-392d-4983-a1e8-8314038ae1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:55.340825 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e9e0c-392d-4983-a1e8-8314038ae1c9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:55.341105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e9e0c-392d-4983-a1e8-8314038ae1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:55.341383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e9e0c-392d-4983-a1e8-8314038ae1c9 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:55.342125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e9e0c-392d-4983-a1e8-8314038ae1c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:55.342433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e9e0c-392d-4983-a1e8-8314038ae1c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:55.343899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e9e0c-392d-4983-a1e8-8314038ae1c9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:55.345419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:55.368231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 512/1030] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 08:33:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:55.369793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949e95c-a74d-49be-826f-4a4454309167 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:55.409983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:55.466785 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e9e0c-392d-4983-a1e8-8314038ae1c9 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:55.471237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949e95c-a74d-49be-826f-4a4454309167 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:55.508417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47eb2d-f6ae-40fe-9345-377a63679651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:55.509023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47eb2d-f6ae-40fe-9345-377a63679651 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:55.509302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47eb2d-f6ae-40fe-9345-377a63679651 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:55.509572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47eb2d-f6ae-40fe-9345-377a63679651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:55.572832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e9e0c-392d-4983-a1e8-8314038ae1c9 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:55.581509 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47eb2d-f6ae-40fe-9345-377a63679651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:55.583436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47eb2d-f6ae-40fe-9345-377a63679651 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:55.608741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff27b9f-ffd3-4f13-9720-f7e65bb01cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:55.609678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff27b9f-ffd3-4f13-9720-f7e65bb01cba None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:55.610124 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff27b9f-ffd3-4f13-9720-f7e65bb01cba None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:55.610402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff27b9f-ffd3-4f13-9720-f7e65bb01cba None admin] PATH_INFO: `/v3/services/image` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:55.611116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff27b9f-ffd3-4f13-9720-f7e65bb01cba None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:55.612330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff27b9f-ffd3-4f13-9720-f7e65bb01cba None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-8ff27b9f-ffd3-4f13-9720-f7e65bb01cba None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:55.622512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:55.623518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:55.624387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 513/1031] 10.222.0.26 () {56 vars in 1105 bytes} [Tue Oct 4 08:33:55 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:55.626170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:55.629611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff27b9f-ffd3-4f13-9720-f7e65bb01cba None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:55.696572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 519/1032] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:33:55 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:33:55.701966 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:55.702581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47eb2d-f6ae-40fe-9345-377a63679651 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:55.756278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff27b9f-ffd3-4f13-9720-f7e65bb01cba None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:55.793151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10925e50-2b1a-4b01-94be-71b67554df5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:55.794013 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10925e50-2b1a-4b01-94be-71b67554df5b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:55.794142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10925e50-2b1a-4b01-94be-71b67554df5b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:55.794319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10925e50-2b1a-4b01-94be-71b67554df5b None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:55.794918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10925e50-2b1a-4b01-94be-71b67554df5b None admin] RBAC: Adding query filter params (name=image) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:55.795225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10925e50-2b1a-4b01-94be-71b67554df5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:55.796680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10925e50-2b1a-4b01-94be-71b67554df5b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:55.802406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 514/1033] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:55.803836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47eb2d-f6ae-40fe-9345-377a63679651 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:55.808207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10925e50-2b1a-4b01-94be-71b67554df5b None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:55.839017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9bf2b-9a85-48c6-b14c-4918250fae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:55.839644 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9bf2b-9a85-48c6-b14c-4918250fae2a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:55.839919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9bf2b-9a85-48c6-b14c-4918250fae2a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:55.840198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9bf2b-9a85-48c6-b14c-4918250fae2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:55.847538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:55.852270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9bf2b-9a85-48c6-b14c-4918250fae2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:55.854407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9bf2b-9a85-48c6-b14c-4918250fae2a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-a1f9bf2b-9a85-48c6-b14c-4918250fae2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:55.869348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:55.870344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:55.871235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:55.871235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:55.871235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:55.871235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 520/1034] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:55.876091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9bf2b-9a85-48c6-b14c-4918250fae2a None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:55.917224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10925e50-2b1a-4b01-94be-71b67554df5b None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:55.939010 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:55.956968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22738b77-5348-4256-bd2a-352344df8f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:55.957747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738b77-5348-4256-bd2a-352344df8f95 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:55.958025 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738b77-5348-4256-bd2a-352344df8f95 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:55.958285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738b77-5348-4256-bd2a-352344df8f95 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:55.958907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22738b77-5348-4256-bd2a-352344df8f95 None admin] RBAC: Adding query filter params (type=image) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:55.959209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22738b77-5348-4256-bd2a-352344df8f95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:55.960554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22738b77-5348-4256-bd2a-352344df8f95 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:55.970143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 515/1035] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:55.971091 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22738b77-5348-4256-bd2a-352344df8f95 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:55.983641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9bf2b-9a85-48c6-b14c-4918250fae2a None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:56.018392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7a92e-e49b-4c55-adc4-f62d511a601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:56.019017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7a92e-e49b-4c55-adc4-f62d511a601f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:56.019267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7a92e-e49b-4c55-adc4-f62d511a601f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:56.019570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7a92e-e49b-4c55-adc4-f62d511a601f None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:56.020201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7a92e-e49b-4c55-adc4-f62d511a601f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:56.020507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7a92e-e49b-4c55-adc4-f62d511a601f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:56.021918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7a92e-e49b-4c55-adc4-f62d511a601f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:56.034225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 521/1036] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:56.039794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7a92e-e49b-4c55-adc4-f62d511a601f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:56.080357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22738b77-5348-4256-bd2a-352344df8f95 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:56.090554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:56.120631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895ff98-f12f-4243-93d4-ea4df9830d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:56.121245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895ff98-f12f-4243-93d4-ea4df9830d1c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:56.121865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895ff98-f12f-4243-93d4-ea4df9830d1c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:56.122146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895ff98-f12f-4243-93d4-ea4df9830d1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:56.123008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895ff98-f12f-4243-93d4-ea4df9830d1c None admin] RBAC: Adding query filter params (interface=public, service_id=3dd06a87b9ce4681a57270c51373e491, region_id=RegionOne) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:56.123319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895ff98-f12f-4243-93d4-ea4df9830d1c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:56.124687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895ff98-f12f-4243-93d4-ea4df9830d1c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:56.134760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 516/1037] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 08:33:55 2022] GET /identity/v3/endpoints?service_id=3dd06a87b9ce4681a57270c51373e491&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:56.135787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895ff98-f12f-4243-93d4-ea4df9830d1c None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:56.147445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7a92e-e49b-4c55-adc4-f62d511a601f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:56.182790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25526ee5-1e7d-4204-9edd-7e8af4d1ca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:56.183415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25526ee5-1e7d-4204-9edd-7e8af4d1ca09 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:56.183694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25526ee5-1e7d-4204-9edd-7e8af4d1ca09 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:56.183965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25526ee5-1e7d-4204-9edd-7e8af4d1ca09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:56.190182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:56.191391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25526ee5-1e7d-4204-9edd-7e8af4d1ca09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:56.192827 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25526ee5-1e7d-4204-9edd-7e8af4d1ca09 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-25526ee5-1e7d-4204-9edd-7e8af4d1ca09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:33:56.204060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:56.204937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:56.205849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:33:56.205849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:33:56.205849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:33:56.205849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:33:56.205849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:33:56.205849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:33:56.205849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:33:56.205849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:56.205849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 522/1038] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:33:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:56.210954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25526ee5-1e7d-4204-9edd-7e8af4d1ca09 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:56.252883 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895ff98-f12f-4243-93d4-ea4df9830d1c None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:56.289220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24e85a-e713-462f-9ab6-bc307b30249c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:56.289852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24e85a-e713-462f-9ab6-bc307b30249c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:56.290129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24e85a-e713-462f-9ab6-bc307b30249c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:56.290400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24e85a-e713-462f-9ab6-bc307b30249c None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:56.290995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24e85a-e713-462f-9ab6-bc307b30249c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:56.292380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24e85a-e713-462f-9ab6-bc307b30249c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:56.309728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 517/1039] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 08:33:56 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:33:56.331840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25526ee5-1e7d-4204-9edd-7e8af4d1ca09 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:56.344951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:56.372451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df192713-81e5-42fc-a827-20aaee165c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:56.373076 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df192713-81e5-42fc-a827-20aaee165c82 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:56.373358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df192713-81e5-42fc-a827-20aaee165c82 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:56.373728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df192713-81e5-42fc-a827-20aaee165c82 None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:56.374453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df192713-81e5-42fc-a827-20aaee165c82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:56.374762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df192713-81e5-42fc-a827-20aaee165c82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:56.376243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df192713-81e5-42fc-a827-20aaee165c82 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:56.390594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 523/1040] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:33:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:56.396810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24e85a-e713-462f-9ab6-bc307b30249c None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:56.509946 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24e85a-e713-462f-9ab6-bc307b30249c None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:56.523935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:56.546090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b26fc9-67fc-4e28-8822-d28130526f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:56.546867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b26fc9-67fc-4e28-8822-d28130526f91 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:56.547148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b26fc9-67fc-4e28-8822-d28130526f91 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:56.547421 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b26fc9-67fc-4e28-8822-d28130526f91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:56.548160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b26fc9-67fc-4e28-8822-d28130526f91 None admin] RBAC: Adding query filter params (role.id=9b17208db146464c84a15bc97da278d9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=ee8a19f4d9704408a8b737095e5885f1) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:56.548490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b26fc9-67fc-4e28-8822-d28130526f91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:56.550001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b26fc9-67fc-4e28-8822-d28130526f91 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:56.551925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e470546-1cd4-4bf8-a545-d50cbecfb714 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:56.552218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e470546-1cd4-4bf8-a545-d50cbecfb714 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:56.552495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e470546-1cd4-4bf8-a545-d50cbecfb714 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:56.553839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 524/1041] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:33:56.556611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 518/1042] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:33:56 2022] GET /identity/v3/role_assignments?user.id=ee8a19f4d9704408a8b737095e5885f1&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=9b17208db146464c84a15bc97da278d9 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:56.558803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da02ba-b5ba-4f52-bec7-22e9517f5241 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:56.559088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da02ba-b5ba-4f52-bec7-22e9517f5241 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:56.559359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da02ba-b5ba-4f52-bec7-22e9517f5241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:56.579064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-a4da02ba-b5ba-4f52-bec7-22e9517f5241 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:56.580280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a4da02ba-b5ba-4f52-bec7-22e9517f5241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:56.666754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 525/1043] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:56.937125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b26fc9-67fc-4e28-8822-d28130526f91 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:57.037918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b26fc9-67fc-4e28-8822-d28130526f91 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:57.071556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5fd88-16cc-418d-8916-540c636a3ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:57.072193 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5fd88-16cc-418d-8916-540c636a3ae2 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:57.072471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5fd88-16cc-418d-8916-540c636a3ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:57.072747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5fd88-16cc-418d-8916-540c636a3ae2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:57.073876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5fd88-16cc-418d-8916-540c636a3ae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:57.075311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5fd88-16cc-418d-8916-540c636a3ae2 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:57.076881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 519/1044] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:57.082712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da02ba-b5ba-4f52-bec7-22e9517f5241 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:57.180093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:57.183965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da02ba-b5ba-4f52-bec7-22e9517f5241 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:57.220974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15784c64-8497-4027-8625-f51d9235e913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:57.221500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15784c64-8497-4027-8625-f51d9235e913 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:57.221810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15784c64-8497-4027-8625-f51d9235e913 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:57.222088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15784c64-8497-4027-8625-f51d9235e913 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:57.223332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15784c64-8497-4027-8625-f51d9235e913 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:57.224770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15784c64-8497-4027-8625-f51d9235e913 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:57.246431 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 526/1045] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:33:57 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:33:58.462799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235da5c-6f4e-402c-9a10-868862c54103 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:58.463320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235da5c-6f4e-402c-9a10-868862c54103 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:58.463320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235da5c-6f4e-402c-9a10-868862c54103 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:58.464816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 520/1046] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:58.469953 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23277c34-d454-4930-a3bb-dfcebb586a15 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:58.470352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23277c34-d454-4930-a3bb-dfcebb586a15 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:58.470512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23277c34-d454-4930-a3bb-dfcebb586a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:58.489999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-23277c34-d454-4930-a3bb-dfcebb586a15 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:58.491185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-23277c34-d454-4930-a3bb-dfcebb586a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:58.559813 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 527/1047] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:58.679423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37ffac-8999-42c2-a201-23695f2bc713 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:58.679700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37ffac-8999-42c2-a201-23695f2bc713 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:58.679975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37ffac-8999-42c2-a201-23695f2bc713 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:58.681436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 521/1048] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:58.686261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f123378-d4d9-4279-af20-258a3f99350c None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:58.686555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f123378-d4d9-4279-af20-258a3f99350c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:58.686831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f123378-d4d9-4279-af20-258a3f99350c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:58.705646 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-4f123378-d4d9-4279-af20-258a3f99350c None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:58.706919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4f123378-d4d9-4279-af20-258a3f99350c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:58.774329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 528/1049] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:33:58.824814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37ffac-8999-42c2-a201-23695f2bc713 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:58.872100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:58.927712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37ffac-8999-42c2-a201-23695f2bc713 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:58.960104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9e768-59d4-424c-b921-b4434c5ec988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:58.960787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9e768-59d4-424c-b921-b4434c5ec988 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:58.961061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9e768-59d4-424c-b921-b4434c5ec988 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:58.961365 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9e768-59d4-424c-b921-b4434c5ec988 None admin] PATH_INFO: `/v3/services/image` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:58.961986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9e768-59d4-424c-b921-b4434c5ec988 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:58.963343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9e768-59d4-424c-b921-b4434c5ec988 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-c2c9e768-59d4-424c-b921-b4434c5ec988 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:58.971991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:58.973041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 08:33:58.974383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:58.978011 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 522/1050] 10.222.0.26 () {56 vars in 1105 bytes} [Tue Oct 4 08:33:58 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:58.979299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f123378-d4d9-4279-af20-258a3f99350c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:58.997756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:59.043979 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9e768-59d4-424c-b921-b4434c5ec988 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:59.085225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f123378-d4d9-4279-af20-258a3f99350c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:59.121111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0380ae-4b56-42c3-b5db-99d4855f2b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:59.121735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0380ae-4b56-42c3-b5db-99d4855f2b6f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:59.122020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0380ae-4b56-42c3-b5db-99d4855f2b6f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:59.122290 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0380ae-4b56-42c3-b5db-99d4855f2b6f None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:59.122913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0380ae-4b56-42c3-b5db-99d4855f2b6f None admin] RBAC: Adding query filter params (name=image) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:59.123213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0380ae-4b56-42c3-b5db-99d4855f2b6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:59.124577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0380ae-4b56-42c3-b5db-99d4855f2b6f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:59.130675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 529/1051] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:33:59.135999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0380ae-4b56-42c3-b5db-99d4855f2b6f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:59.153227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9e768-59d4-424c-b921-b4434c5ec988 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:59.164196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:59.190622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954aff4b-8106-4099-bb3d-521034c17c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:59.191911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954aff4b-8106-4099-bb3d-521034c17c99 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:59.192248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954aff4b-8106-4099-bb3d-521034c17c99 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:59.192579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954aff4b-8106-4099-bb3d-521034c17c99 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:59.202704 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954aff4b-8106-4099-bb3d-521034c17c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-954aff4b-8106-4099-bb3d-521034c17c99 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:33:59.207455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:59.208392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:59.209596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 523/1052] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Oct 4 08:33:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:59.214263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954aff4b-8106-4099-bb3d-521034c17c99 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:59.229552 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:59.260263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0380ae-4b56-42c3-b5db-99d4855f2b6f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:59.295678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5519d-09be-4221-b0c0-420c01e65505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:59.296198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5519d-09be-4221-b0c0-420c01e65505 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:59.296475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5519d-09be-4221-b0c0-420c01e65505 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:59.296744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5519d-09be-4221-b0c0-420c01e65505 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:59.297365 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5519d-09be-4221-b0c0-420c01e65505 None admin] RBAC: Adding query filter params (type=image) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:59.297665 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5519d-09be-4221-b0c0-420c01e65505 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:59.299293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5519d-09be-4221-b0c0-420c01e65505 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:59.304493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 530/1053] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:59.309825 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5519d-09be-4221-b0c0-420c01e65505 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:59.329451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954aff4b-8106-4099-bb3d-521034c17c99 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:59.367577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f1fdf-3370-4af5-b7b8-f6d883db895d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:59.368172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f1fdf-3370-4af5-b7b8-f6d883db895d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:59.368453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f1fdf-3370-4af5-b7b8-f6d883db895d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:59.368723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f1fdf-3370-4af5-b7b8-f6d883db895d None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:59.369382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f1fdf-3370-4af5-b7b8-f6d883db895d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:59.369680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f1fdf-3370-4af5-b7b8-f6d883db895d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:59.371057 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f1fdf-3370-4af5-b7b8-f6d883db895d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:59.380219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 524/1054] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 08:33:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:59.385401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f1fdf-3370-4af5-b7b8-f6d883db895d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:59.418576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:59.435448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5519d-09be-4221-b0c0-420c01e65505 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:59.469954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3fe8d-0934-461b-bb29-5c83a99219b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:59.470651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3fe8d-0934-461b-bb29-5c83a99219b3 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:59.470866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3fe8d-0934-461b-bb29-5c83a99219b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:59.471138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3fe8d-0934-461b-bb29-5c83a99219b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:59.471807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3fe8d-0934-461b-bb29-5c83a99219b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:59.473804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3fe8d-0934-461b-bb29-5c83a99219b3 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:59.489122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:59.489841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f1fdf-3370-4af5-b7b8-f6d883db895d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:59.494698 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 531/1055] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:33:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:33:59.495952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbaa09-8929-495c-a234-a0748da025aa None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:59.496302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbaa09-8929-495c-a234-a0748da025aa None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:59.496795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbaa09-8929-495c-a234-a0748da025aa None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:59.498458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 532/1056] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:33:59.507846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de23d00-d16b-4df5-b2cd-a47c590c2d13 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:59.508165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de23d00-d16b-4df5-b2cd-a47c590c2d13 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:59.508397 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de23d00-d16b-4df5-b2cd-a47c590c2d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:59.528392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-2de23d00-d16b-4df5-b2cd-a47c590c2d13 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:33:59.529638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2de23d00-d16b-4df5-b2cd-a47c590c2d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:33:59.547080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250e3fc-24da-4c0d-82e0-dff892bd8ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:59.547632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250e3fc-24da-4c0d-82e0-dff892bd8ed6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:59.547913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250e3fc-24da-4c0d-82e0-dff892bd8ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:59.548179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250e3fc-24da-4c0d-82e0-dff892bd8ed6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:59.560764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250e3fc-24da-4c0d-82e0-dff892bd8ed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:59.562510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250e3fc-24da-4c0d-82e0-dff892bd8ed6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-5250e3fc-24da-4c0d-82e0-dff892bd8ed6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:33:59.579348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:59.580446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:33:59.581361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:33:59.581361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:33:59.581361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:33:59.581361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 525/1057] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:33:59.587243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250e3fc-24da-4c0d-82e0-dff892bd8ed6 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:59.611208 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 533/1058] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:33:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:33:59.683802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:59.707916 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250e3fc-24da-4c0d-82e0-dff892bd8ed6 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:59.742700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde8802-b8e4-45a7-b864-33750c05efaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:59.743307 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde8802-b8e4-45a7-b864-33750c05efaf None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:59.743588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde8802-b8e4-45a7-b864-33750c05efaf None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:59.743856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde8802-b8e4-45a7-b864-33750c05efaf None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:59.744490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde8802-b8e4-45a7-b864-33750c05efaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:33:59.744794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde8802-b8e4-45a7-b864-33750c05efaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:59.746182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde8802-b8e4-45a7-b864-33750c05efaf None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:59.761760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 526/1059] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:33:59.764829 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de23d00-d16b-4df5-b2cd-a47c590c2d13 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:59.865322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de23d00-d16b-4df5-b2cd-a47c590c2d13 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:33:59.880144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde8802-b8e4-45a7-b864-33750c05efaf None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:59.895844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:33:59.902793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace3603-66ab-41ae-aea5-ccd110a070d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:33:59.903876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace3603-66ab-41ae-aea5-ccd110a070d3 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:33:59.904158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace3603-66ab-41ae-aea5-ccd110a070d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:33:59.904582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace3603-66ab-41ae-aea5-ccd110a070d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:33:59.911878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace3603-66ab-41ae-aea5-ccd110a070d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:33:59.913305 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace3603-66ab-41ae-aea5-ccd110a070d3 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-4ace3603-66ab-41ae-aea5-ccd110a070d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:33:59.924623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:33:59.925871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:33:59.926989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:33:59.926989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:33:59.926989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:33:59.926989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:33:59.926989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:33:59.926989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:33:59.926989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:33:59.926989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:33:59.926989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:33:59.926989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:33:59.926989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 534/1060] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 08:33:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:33:59.931330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace3603-66ab-41ae-aea5-ccd110a070d3 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:33:59.995252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde8802-b8e4-45a7-b864-33750c05efaf None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:00.033329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f8f23-a363-46e2-abd6-0e11f4cebd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:00.033650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace3603-66ab-41ae-aea5-ccd110a070d3 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:00.034533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f8f23-a363-46e2-abd6-0e11f4cebd7a None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:00.035507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f8f23-a363-46e2-abd6-0e11f4cebd7a None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:00.035886 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f8f23-a363-46e2-abd6-0e11f4cebd7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:00.042918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:00.047093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f8f23-a363-46e2-abd6-0e11f4cebd7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-916f8f23-a363-46e2-abd6-0e11f4cebd7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:34:00.053588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:00.054652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:00.055497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:00.055497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:00.055497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:00.055497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:00.055497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:34:00.055497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:34:00.055497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:34:00.055497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:34:00.055497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:34:00.055497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:34:00.055497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:00.055497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:34:00.056150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 527/1061] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:33:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:34:00.061205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f8f23-a363-46e2-abd6-0e11f4cebd7a None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:00.072288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716a8fd-5fc8-40da-9146-23ecd455ad04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:00.073340 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716a8fd-5fc8-40da-9146-23ecd455ad04 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:00.073705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716a8fd-5fc8-40da-9146-23ecd455ad04 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:00.074069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716a8fd-5fc8-40da-9146-23ecd455ad04 None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:00.075197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716a8fd-5fc8-40da-9146-23ecd455ad04 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:00.075711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716a8fd-5fc8-40da-9146-23ecd455ad04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:00.077420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716a8fd-5fc8-40da-9146-23ecd455ad04 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:00.099869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 535/1062] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 08:33:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:00.105391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716a8fd-5fc8-40da-9146-23ecd455ad04 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:00.154918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:00.170427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f8f23-a363-46e2-abd6-0e11f4cebd7a None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:00.204894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716a8fd-5fc8-40da-9146-23ecd455ad04 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:00.206986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378b042-00de-4d65-b3aa-4d5a02b1f28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:00.207703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378b042-00de-4d65-b3aa-4d5a02b1f28f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:00.207984 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378b042-00de-4d65-b3aa-4d5a02b1f28f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:00.208303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378b042-00de-4d65-b3aa-4d5a02b1f28f None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:00.208959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378b042-00de-4d65-b3aa-4d5a02b1f28f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:00.209257 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378b042-00de-4d65-b3aa-4d5a02b1f28f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:00.210637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378b042-00de-4d65-b3aa-4d5a02b1f28f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:00.224397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 528/1063] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:34:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:34:00.225398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378b042-00de-4d65-b3aa-4d5a02b1f28f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:00.243250 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a6a59-e8cf-4607-a8bd-e99ae3351eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:00.243970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a6a59-e8cf-4607-a8bd-e99ae3351eeb None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:00.244319 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a6a59-e8cf-4607-a8bd-e99ae3351eeb None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:00.244657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a6a59-e8cf-4607-a8bd-e99ae3351eeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:00.255285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a6a59-e8cf-4607-a8bd-e99ae3351eeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:00.256822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a6a59-e8cf-4607-a8bd-e99ae3351eeb None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-e82a6a59-e8cf-4607-a8bd-e99ae3351eeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:34:00.271869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:00.273026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:34:00.273989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:34:00.273989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:34:00.273989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:34:00.273989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 536/1064] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:34:00.278486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a6a59-e8cf-4607-a8bd-e99ae3351eeb None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:00.294851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:00.340669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378b042-00de-4d65-b3aa-4d5a02b1f28f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:00.374808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9d5b4-2ce9-4921-9b48-1f3cd370ffe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:00.375441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9d5b4-2ce9-4921-9b48-1f3cd370ffe7 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:00.375721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9d5b4-2ce9-4921-9b48-1f3cd370ffe7 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:00.375998 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9d5b4-2ce9-4921-9b48-1f3cd370ffe7 None admin] PATH_INFO: `/v3/users/37b3d81d52434116be7c0238855bd243` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:00.383076 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:00.388037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a6a59-e8cf-4607-a8bd-e99ae3351eeb None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:00.389662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9d5b4-2ce9-4921-9b48-1f3cd370ffe7 None admin] RBAC: Authorizing `identity:get_user(user_id=37b3d81d52434116be7c0238855bd243)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:00.391129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9d5b4-2ce9-4921-9b48-1f3cd370ffe7 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:00.397949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 529/1065] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:00 2022] GET /identity/v3/users/37b3d81d52434116be7c0238855bd243 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:00.399203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9d5b4-2ce9-4921-9b48-1f3cd370ffe7 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:00.428537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af28cfc-5720-4733-823b-e15ee7c5adee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:00.429158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af28cfc-5720-4733-823b-e15ee7c5adee None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:00.429512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af28cfc-5720-4733-823b-e15ee7c5adee None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:00.429889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af28cfc-5720-4733-823b-e15ee7c5adee None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:00.430598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af28cfc-5720-4733-823b-e15ee7c5adee None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:00.430971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af28cfc-5720-4733-823b-e15ee7c5adee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:00.432473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af28cfc-5720-4733-823b-e15ee7c5adee None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:00.444731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 537/1066] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:00.450406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af28cfc-5720-4733-823b-e15ee7c5adee None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:00.505941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9d5b4-2ce9-4921-9b48-1f3cd370ffe7 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:00.524399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:00.542324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b5a57-1791-4c8a-ade8-ac83c582ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:00.542929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b5a57-1791-4c8a-ade8-ac83c582ac4d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:00.543209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b5a57-1791-4c8a-ade8-ac83c582ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:00.543485 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b5a57-1791-4c8a-ade8-ac83c582ac4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:00.565902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af28cfc-5720-4733-823b-e15ee7c5adee None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:00.602201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58293049-2d76-42e0-9f75-bb2d5afac181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:00.603019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58293049-2d76-42e0-9f75-bb2d5afac181 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:00.603724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58293049-2d76-42e0-9f75-bb2d5afac181 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:00.604155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58293049-2d76-42e0-9f75-bb2d5afac181 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:00.611815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58293049-2d76-42e0-9f75-bb2d5afac181 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:00.613487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58293049-2d76-42e0-9f75-bb2d5afac181 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:00.614249 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b5a57-1791-4c8a-ade8-ac83c582ac4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:00.615681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851b5a57-1791-4c8a-ade8-ac83c582ac4d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-58293049-2d76-42e0-9f75-bb2d5afac181 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:34:00.625084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:00.626155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:00.627257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:34:00.627257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:34:00.627257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:34:00.627257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:34:00.627257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:34:00.627257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:34:00.627257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:34:00.627257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:34:00.629434 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 538/1067] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:34:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:34:00.635029 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58293049-2d76-42e0-9f75-bb2d5afac181 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:00.658042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:00.737228 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 530/1068] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:34:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:34:00.743578 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b5a57-1791-4c8a-ade8-ac83c582ac4d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:00.752241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58293049-2d76-42e0-9f75-bb2d5afac181 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:00.761260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:00.789964 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631311d-6c3c-48fd-a060-b9efa7dc1a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:00.790600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631311d-6c3c-48fd-a060-b9efa7dc1a87 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:00.790879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631311d-6c3c-48fd-a060-b9efa7dc1a87 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:00.791197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631311d-6c3c-48fd-a060-b9efa7dc1a87 None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:00.791844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631311d-6c3c-48fd-a060-b9efa7dc1a87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:00.792151 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631311d-6c3c-48fd-a060-b9efa7dc1a87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:00.793592 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631311d-6c3c-48fd-a060-b9efa7dc1a87 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:00.805724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 539/1069] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:34:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:34:00.811482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631311d-6c3c-48fd-a060-b9efa7dc1a87 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:00.853847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851b5a57-1791-4c8a-ade8-ac83c582ac4d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:00.889811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340cba2-e4d5-4705-a13f-25a755cf612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:00.890536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340cba2-e4d5-4705-a13f-25a755cf612f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:00.890876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340cba2-e4d5-4705-a13f-25a755cf612f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:00.891207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340cba2-e4d5-4705-a13f-25a755cf612f None admin] PATH_INFO: `/v3/projects/c896707714b141faa1c51f1716574cb1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:00.892370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340cba2-e4d5-4705-a13f-25a755cf612f None admin] RBAC: Authorizing `identity:get_project(project_id=c896707714b141faa1c51f1716574cb1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:00.893921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340cba2-e4d5-4705-a13f-25a755cf612f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:00.896487 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 531/1070] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:34:00 2022] GET /identity/v3/projects/c896707714b141faa1c51f1716574cb1 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:34:00.901932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:00.903030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340cba2-e4d5-4705-a13f-25a755cf612f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:00.921594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631311d-6c3c-48fd-a060-b9efa7dc1a87 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:00.956784 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae968a7a-4f4e-4954-a2c9-52b15617518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:00.957641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae968a7a-4f4e-4954-a2c9-52b15617518b None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:00.957951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae968a7a-4f4e-4954-a2c9-52b15617518b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:00.958230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae968a7a-4f4e-4954-a2c9-52b15617518b None admin] PATH_INFO: `/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/ee8a19f4d9704408a8b737095e5885f1/roles/9b17208db146464c84a15bc97da278d9` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:00.960359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae968a7a-4f4e-4954-a2c9-52b15617518b None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3df14a62714f6bab2acdfc979b655b, user_id=ee8a19f4d9704408a8b737095e5885f1, role_id=9b17208db146464c84a15bc97da278d9)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:00.961831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae968a7a-4f4e-4954-a2c9-52b15617518b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:00.981724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 540/1071] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:34:00 2022] PUT /identity/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/ee8a19f4d9704408a8b737095e5885f1/roles/9b17208db146464c84a15bc97da278d9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:34:01.004877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:01.021344 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340cba2-e4d5-4705-a13f-25a755cf612f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:01.057903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e0c61-03b0-4c15-a5ca-5a4b3dac8220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:01.058779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e0c61-03b0-4c15-a5ca-5a4b3dac8220 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:01.059260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e0c61-03b0-4c15-a5ca-5a4b3dac8220 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:01.059689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e0c61-03b0-4c15-a5ca-5a4b3dac8220 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:01.060589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e0c61-03b0-4c15-a5ca-5a4b3dac8220 None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=c896707714b141faa1c51f1716574cb1, user.id=37b3d81d52434116be7c0238855bd243) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:01.061208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e0c61-03b0-4c15-a5ca-5a4b3dac8220 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:01.063173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e0c61-03b0-4c15-a5ca-5a4b3dac8220 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:01.075858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 532/1072] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:34:00 2022] GET /identity/v3/role_assignments?user.id=37b3d81d52434116be7c0238855bd243&scope.project.id=c896707714b141faa1c51f1716574cb1&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:01.686750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1bb0c-4854-43fe-8d9e-efdf4e9d5efb None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:01.687284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1bb0c-4854-43fe-8d9e-efdf4e9d5efb None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:01.687284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1bb0c-4854-43fe-8d9e-efdf4e9d5efb None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:01.688763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 541/1073] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:01.693832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19106c-cb5f-4c3c-86b1-1957b7b19e86 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:01.694210 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19106c-cb5f-4c3c-86b1-1957b7b19e86 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:01.694402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19106c-cb5f-4c3c-86b1-1957b7b19e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:01.712846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-8c19106c-cb5f-4c3c-86b1-1957b7b19e86 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:01.714075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8c19106c-cb5f-4c3c-86b1-1957b7b19e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:01.782248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 533/1074] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:02.046468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1bb0c-4854-43fe-8d9e-efdf4e9d5efb None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:02.146534 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1bb0c-4854-43fe-8d9e-efdf4e9d5efb None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:02.182830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057b02-0751-4dd8-bfee-3b7d6d8d5e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:02.183432 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057b02-0751-4dd8-bfee-3b7d6d8d5e84 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:02.183711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057b02-0751-4dd8-bfee-3b7d6d8d5e84 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:02.183983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057b02-0751-4dd8-bfee-3b7d6d8d5e84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:02.194717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:02.254412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057b02-0751-4dd8-bfee-3b7d6d8d5e84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:02.255881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057b02-0751-4dd8-bfee-3b7d6d8d5e84 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:02.354889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 542/1075] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:34:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:34:02.360594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19106c-cb5f-4c3c-86b1-1957b7b19e86 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:02.440802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:02.461096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19106c-cb5f-4c3c-86b1-1957b7b19e86 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:02.494257 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97faedbc-0a9f-45eb-bb76-f2e0d8cd9972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:02.494953 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faedbc-0a9f-45eb-bb76-f2e0d8cd9972 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:02.495302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faedbc-0a9f-45eb-bb76-f2e0d8cd9972 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:02.495643 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faedbc-0a9f-45eb-bb76-f2e0d8cd9972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:02.505445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faedbc-0a9f-45eb-bb76-f2e0d8cd9972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:02.506974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faedbc-0a9f-45eb-bb76-f2e0d8cd9972 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-97faedbc-0a9f-45eb-bb76-f2e0d8cd9972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:34:02.521552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:02.522699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:34:02.523805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:34:02.523805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:34:02.523805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:34:02.524587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 534/1076] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:34:02.530363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057b02-0751-4dd8-bfee-3b7d6d8d5e84 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:02.571923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:02.631359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057b02-0751-4dd8-bfee-3b7d6d8d5e84 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:02.675536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e0cad-346a-4f71-bbfa-27e78d4cdbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:02.676148 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e0cad-346a-4f71-bbfa-27e78d4cdbde None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:02.676429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e0cad-346a-4f71-bbfa-27e78d4cdbde None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:02.676700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e0cad-346a-4f71-bbfa-27e78d4cdbde None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:02.677342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e0cad-346a-4f71-bbfa-27e78d4cdbde None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:02.677647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e0cad-346a-4f71-bbfa-27e78d4cdbde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:02.679040 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e0cad-346a-4f71-bbfa-27e78d4cdbde None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:02.697740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 543/1077] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:03.155169 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad62f6b-ce25-4f5c-8377-3ed75680ff5c None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:03.155695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad62f6b-ce25-4f5c-8377-3ed75680ff5c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:03.155695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad62f6b-ce25-4f5c-8377-3ed75680ff5c None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:03.157077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 535/1078] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:03.162366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c145b25-1988-4ee7-a643-cab2250d493b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:03.162659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c145b25-1988-4ee7-a643-cab2250d493b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:03.162932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c145b25-1988-4ee7-a643-cab2250d493b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:03.181176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-2c145b25-1988-4ee7-a643-cab2250d493b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:03.182383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2c145b25-1988-4ee7-a643-cab2250d493b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:03.235811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b2e4d-e023-4f02-b553-b37505c4e0fa None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:03.236095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b2e4d-e023-4f02-b553-b37505c4e0fa None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:03.236379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b2e4d-e023-4f02-b553-b37505c4e0fa None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:03.238370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 536/1079] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:34:03.245669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c1f05-b9a7-433b-b248-11ade966bb33 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:03.245991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c1f05-b9a7-433b-b248-11ade966bb33 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:03.246266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c1f05-b9a7-433b-b248-11ade966bb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:03.249088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 544/1080] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:03.264597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-484c1f05-b9a7-433b-b248-11ade966bb33 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:03.266027 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-484c1f05-b9a7-433b-b248-11ade966bb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:03.330821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 537/1081] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 08:34:03.517941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c145b25-1988-4ee7-a643-cab2250d493b None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:03.546930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:03.598250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c1f05-b9a7-433b-b248-11ade966bb33 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:03.618450 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c145b25-1988-4ee7-a643-cab2250d493b None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:03.652892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ec3d1-0c9f-46df-b0e4-376c6d3776e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:03.653523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ec3d1-0c9f-46df-b0e4-376c6d3776e5 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:03.653835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ec3d1-0c9f-46df-b0e4-376c6d3776e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:03.654119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ec3d1-0c9f-46df-b0e4-376c6d3776e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:03.664469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ec3d1-0c9f-46df-b0e4-376c6d3776e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-d38ec3d1-0c9f-46df-b0e4-376c6d3776e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:34:03.669162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:03.670163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:34:03.671001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 545/1082] 10.222.0.26 () {56 vars in 1120 bytes} [Tue Oct 4 08:34:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:34:03.672287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:03.676220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ec3d1-0c9f-46df-b0e4-376c6d3776e5 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:03.710314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c1f05-b9a7-433b-b248-11ade966bb33 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:03.742739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5e674-491b-45b0-970d-e732c20d7dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:03.743363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5e674-491b-45b0-970d-e732c20d7dcd None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:03.743638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5e674-491b-45b0-970d-e732c20d7dcd None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:03.743907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5e674-491b-45b0-970d-e732c20d7dcd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:03.754153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5e674-491b-45b0-970d-e732c20d7dcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-aea5e674-491b-45b0-970d-e732c20d7dcd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:34:03.758829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:03.759766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:34:03.760610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 538/1083] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:34:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:34:03.766059 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5e674-491b-45b0-970d-e732c20d7dcd None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:03.768215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:03.779283 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ec3d1-0c9f-46df-b0e4-376c6d3776e5 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:03.813502 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c084211-2453-4d0e-a5b0-963b7a84874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:03.814126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c084211-2453-4d0e-a5b0-963b7a84874f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:03.814423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c084211-2453-4d0e-a5b0-963b7a84874f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:03.814678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c084211-2453-4d0e-a5b0-963b7a84874f None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:03.815348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c084211-2453-4d0e-a5b0-963b7a84874f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:03.815632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c084211-2453-4d0e-a5b0-963b7a84874f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:03.817526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c084211-2453-4d0e-a5b0-963b7a84874f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:03.825215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 546/1084] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 08:34:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:34:03.830942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c084211-2453-4d0e-a5b0-963b7a84874f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:03.865544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5e674-491b-45b0-970d-e732c20d7dcd None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:03.887720 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:03.900106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2cc1a6-de76-4a8e-a3ca-fd576b545c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:03.900708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cc1a6-de76-4a8e-a3ca-fd576b545c63 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:03.900987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cc1a6-de76-4a8e-a3ca-fd576b545c63 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:03.901260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2cc1a6-de76-4a8e-a3ca-fd576b545c63 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:03.901952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2cc1a6-de76-4a8e-a3ca-fd576b545c63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:03.902259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2cc1a6-de76-4a8e-a3ca-fd576b545c63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:03.903620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2cc1a6-de76-4a8e-a3ca-fd576b545c63 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:03.911617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 539/1085] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:34:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:34:03.917421 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2cc1a6-de76-4a8e-a3ca-fd576b545c63 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:03.931608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c084211-2453-4d0e-a5b0-963b7a84874f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:03.967797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffb296-eea1-43f4-8ba7-e42fab47062e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:03.968423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffb296-eea1-43f4-8ba7-e42fab47062e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:03.968704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffb296-eea1-43f4-8ba7-e42fab47062e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:03.968977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffb296-eea1-43f4-8ba7-e42fab47062e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:03.980216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffb296-eea1-43f4-8ba7-e42fab47062e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:03.981674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffb296-eea1-43f4-8ba7-e42fab47062e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-a6ffb296-eea1-43f4-8ba7-e42fab47062e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:34:03.995877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:03.996906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:34:03.997821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:34:03.997821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:34:03.997821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:34:03.997821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 547/1086] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:34:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:34:04.002902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffb296-eea1-43f4-8ba7-e42fab47062e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:04.006861 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:04.023658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2cc1a6-de76-4a8e-a3ca-fd576b545c63 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:04.056639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b395f-257e-45ea-af0c-6d8afb620b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:04.057340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b395f-257e-45ea-af0c-6d8afb620b0f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:04.057615 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b395f-257e-45ea-af0c-6d8afb620b0f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:04.057921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b395f-257e-45ea-af0c-6d8afb620b0f None admin] PATH_INFO: `/v3/users/37b3d81d52434116be7c0238855bd243` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:04.059044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b395f-257e-45ea-af0c-6d8afb620b0f None admin] RBAC: Authorizing `identity:get_user(user_id=37b3d81d52434116be7c0238855bd243)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:04.060456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b395f-257e-45ea-af0c-6d8afb620b0f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:04.062045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 540/1087] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:03 2022] GET /identity/v3/users/37b3d81d52434116be7c0238855bd243 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:34:04.067856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b395f-257e-45ea-af0c-6d8afb620b0f None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:04.105043 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffb296-eea1-43f4-8ba7-e42fab47062e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:04.118509 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:04.138447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62fada-5607-47ed-8654-49746c2739ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:04.139045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62fada-5607-47ed-8654-49746c2739ef None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:04.139322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62fada-5607-47ed-8654-49746c2739ef None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:04.139592 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62fada-5607-47ed-8654-49746c2739ef None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:04.140221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62fada-5607-47ed-8654-49746c2739ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:04.140520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62fada-5607-47ed-8654-49746c2739ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:04.141920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62fada-5607-47ed-8654-49746c2739ef None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:04.153617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 548/1088] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:34:04.159108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62fada-5607-47ed-8654-49746c2739ef None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:04.165051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b395f-257e-45ea-af0c-6d8afb620b0f None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:04.199166 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc70cd-2d5a-4583-9f1b-c320ea29c423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:04.199787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc70cd-2d5a-4583-9f1b-c320ea29c423 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:04.200074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc70cd-2d5a-4583-9f1b-c320ea29c423 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:04.200343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc70cd-2d5a-4583-9f1b-c320ea29c423 None admin] PATH_INFO: `/v3/projects/c896707714b141faa1c51f1716574cb1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:04.201448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc70cd-2d5a-4583-9f1b-c320ea29c423 None admin] RBAC: Authorizing `identity:get_project(project_id=c896707714b141faa1c51f1716574cb1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:04.202917 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc70cd-2d5a-4583-9f1b-c320ea29c423 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:04.204527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 541/1089] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:34:04 2022] GET /identity/v3/projects/c896707714b141faa1c51f1716574cb1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:34:04.210702 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc70cd-2d5a-4583-9f1b-c320ea29c423 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:04.221210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:04.259905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62fada-5607-47ed-8654-49746c2739ef None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:04.294145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e039f-96c7-4536-ad71-5a6347a506a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:04.294704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e039f-96c7-4536-ad71-5a6347a506a3 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:04.294981 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e039f-96c7-4536-ad71-5a6347a506a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:04.295257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e039f-96c7-4536-ad71-5a6347a506a3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:04.302063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e039f-96c7-4536-ad71-5a6347a506a3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:04.303489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e039f-96c7-4536-ad71-5a6347a506a3 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:04.307099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc70cd-2d5a-4583-9f1b-c320ea29c423 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-035e039f-96c7-4536-ad71-5a6347a506a3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:34:04.314625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:04.315763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:04.317723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:04.317723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:04.317723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:34:04.317723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:34:04.317723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:34:04.317723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:34:04.317723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:34:04.317723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:34:04.317723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 08:34:04.317723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:34:04.317723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 549/1090] 10.222.0.26 () {56 vars in 1093 bytes} [Tue Oct 4 08:34:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:34:04.321893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e039f-96c7-4536-ad71-5a6347a506a3 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:04.328316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:04.342877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33613b4b-1f95-47b8-99c7-3d6c630ea3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:04.343526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33613b4b-1f95-47b8-99c7-3d6c630ea3bf None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:04.343807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33613b4b-1f95-47b8-99c7-3d6c630ea3bf None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:04.344076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33613b4b-1f95-47b8-99c7-3d6c630ea3bf None admin] PATH_INFO: `/v3/projects/c896707714b141faa1c51f1716574cb1/users/37b3d81d52434116be7c0238855bd243/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:04.345965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33613b4b-1f95-47b8-99c7-3d6c630ea3bf None admin] RBAC: Authorizing `identity:create_grant(project_id=c896707714b141faa1c51f1716574cb1, user_id=37b3d81d52434116be7c0238855bd243, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:04.347882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33613b4b-1f95-47b8-99c7-3d6c630ea3bf None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:04.360365 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 542/1091] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:34:04 2022] PUT /identity/v3/projects/c896707714b141faa1c51f1716574cb1/users/37b3d81d52434116be7c0238855bd243/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:34:04.426547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e039f-96c7-4536-ad71-5a6347a506a3 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:04.451561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:04.463482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839facb-6121-4e33-b765-e53bcb38d26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:04.464100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839facb-6121-4e33-b765-e53bcb38d26c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:04.464381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839facb-6121-4e33-b765-e53bcb38d26c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:04.464653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839facb-6121-4e33-b765-e53bcb38d26c None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:04.465392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839facb-6121-4e33-b765-e53bcb38d26c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:04.465710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839facb-6121-4e33-b765-e53bcb38d26c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:04.467181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839facb-6121-4e33-b765-e53bcb38d26c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:04.487457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 550/1092] 10.222.0.26 () {56 vars in 1133 bytes} [Tue Oct 4 08:34:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:04.493542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33613b4b-1f95-47b8-99c7-3d6c630ea3bf None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:04.595463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33613b4b-1f95-47b8-99c7-3d6c630ea3bf None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:04.627605 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3028a9-5d4e-42f6-a9bc-5003773e91a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:04.628206 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3028a9-5d4e-42f6-a9bc-5003773e91a5 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:04.629021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3028a9-5d4e-42f6-a9bc-5003773e91a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:04.629293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3028a9-5d4e-42f6-a9bc-5003773e91a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:04.679345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:04.700715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3028a9-5d4e-42f6-a9bc-5003773e91a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:04.702216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3028a9-5d4e-42f6-a9bc-5003773e91a5 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:04.796253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 543/1093] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:34:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:34:04.802957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839facb-6121-4e33-b765-e53bcb38d26c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:04.899959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839facb-6121-4e33-b765-e53bcb38d26c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:04.932376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f337ef-7181-407d-b556-c8d02df90101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:04.932990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f337ef-7181-407d-b556-c8d02df90101 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:04.933266 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f337ef-7181-407d-b556-c8d02df90101 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:04.933535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f337ef-7181-407d-b556-c8d02df90101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:04.943103 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f337ef-7181-407d-b556-c8d02df90101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:04.944539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f337ef-7181-407d-b556-c8d02df90101 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-a6f337ef-7181-407d-b556-c8d02df90101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:34:04.959047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:04.959983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:34:04.960951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:34:04.960951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:34:04.960951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:34:04.960951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 551/1094] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:34:04.965859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3028a9-5d4e-42f6-a9bc-5003773e91a5 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:05.043756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:05.063550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3028a9-5d4e-42f6-a9bc-5003773e91a5 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:05.095921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d8740-0993-45fb-8d38-7d816348bbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:05.096524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8740-0993-45fb-8d38-7d816348bbe0 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:05.096807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8740-0993-45fb-8d38-7d816348bbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:05.097603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d8740-0993-45fb-8d38-7d816348bbe0 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:05.098271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d8740-0993-45fb-8d38-7d816348bbe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:05.098573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d8740-0993-45fb-8d38-7d816348bbe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:05.099927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d8740-0993-45fb-8d38-7d816348bbe0 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:05.111285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 544/1095] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:05.116809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f337ef-7181-407d-b556-c8d02df90101 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:05.151203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:05.216360 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f337ef-7181-407d-b556-c8d02df90101 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:05.251556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81997215-33c4-4044-b33b-8f2f5beb8c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:05.252472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81997215-33c4-4044-b33b-8f2f5beb8c7f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:05.252748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81997215-33c4-4044-b33b-8f2f5beb8c7f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:05.253017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81997215-33c4-4044-b33b-8f2f5beb8c7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:05.260116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81997215-33c4-4044-b33b-8f2f5beb8c7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:05.261565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81997215-33c4-4044-b33b-8f2f5beb8c7f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-81997215-33c4-4044-b33b-8f2f5beb8c7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:34:05.273007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:05.274090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:05.275014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:34:05.275014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:34:05.275014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:34:05.275014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:34:05.275014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:34:05.275014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:34:05.275014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:34:05.275014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:34:05.275014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 552/1096] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:34:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:34:05.280277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d8740-0993-45fb-8d38-7d816348bbe0 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:05.375881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d8740-0993-45fb-8d38-7d816348bbe0 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:05.408164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c5668-f4d1-41e1-803b-cd273619f67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:05.408702 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c5668-f4d1-41e1-803b-cd273619f67d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:05.408981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c5668-f4d1-41e1-803b-cd273619f67d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:05.409251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c5668-f4d1-41e1-803b-cd273619f67d None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:05.409957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c5668-f4d1-41e1-803b-cd273619f67d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:05.410261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c5668-f4d1-41e1-803b-cd273619f67d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:05.411669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c5668-f4d1-41e1-803b-cd273619f67d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:05.420359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:05.429317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 545/1097] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:34:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:05.430499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81997215-33c4-4044-b33b-8f2f5beb8c7f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:05.521942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:05.528852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81997215-33c4-4044-b33b-8f2f5beb8c7f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:05.561285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114b444-ee43-4f39-bdbc-eb7b2ec5aee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:05.561918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114b444-ee43-4f39-bdbc-eb7b2ec5aee3 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:05.562196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114b444-ee43-4f39-bdbc-eb7b2ec5aee3 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:05.562464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114b444-ee43-4f39-bdbc-eb7b2ec5aee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:05.563173 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114b444-ee43-4f39-bdbc-eb7b2ec5aee3 None admin] RBAC: Adding query filter params (role.id=9b17208db146464c84a15bc97da278d9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=ee8a19f4d9704408a8b737095e5885f1) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:05.563480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114b444-ee43-4f39-bdbc-eb7b2ec5aee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:05.564900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114b444-ee43-4f39-bdbc-eb7b2ec5aee3 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:05.571181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 553/1098] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:34:05 2022] GET /identity/v3/role_assignments?user.id=ee8a19f4d9704408a8b737095e5885f1&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=9b17208db146464c84a15bc97da278d9 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:06.504805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df262c07-9c3e-4765-bd43-3424a6395aae None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:06.505335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df262c07-9c3e-4765-bd43-3424a6395aae None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:06.505335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df262c07-9c3e-4765-bd43-3424a6395aae None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:06.506716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 546/1099] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:06.511943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee12a52-f5da-4667-bf3a-b5e7eeaaa591 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:06.512359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee12a52-f5da-4667-bf3a-b5e7eeaaa591 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:06.512488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee12a52-f5da-4667-bf3a-b5e7eeaaa591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:06.530627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-fee12a52-f5da-4667-bf3a-b5e7eeaaa591 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:06.531785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fee12a52-f5da-4667-bf3a-b5e7eeaaa591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:06.595079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 554/1100] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:06.863378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df262c07-9c3e-4765-bd43-3424a6395aae None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:06.960658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df262c07-9c3e-4765-bd43-3424a6395aae None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:06.993681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a887f-8add-45ba-b65b-d7e9dbe60491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:06.994313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a887f-8add-45ba-b65b-d7e9dbe60491 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:06.994590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a887f-8add-45ba-b65b-d7e9dbe60491 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:06.994861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a887f-8add-45ba-b65b-d7e9dbe60491 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:07.005238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a887f-8add-45ba-b65b-d7e9dbe60491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-504a887f-8add-45ba-b65b-d7e9dbe60491 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:34:07.009934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:07.011045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:34:07.012267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 547/1101] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:34:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:34:07.016731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee12a52-f5da-4667-bf3a-b5e7eeaaa591 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:07.098573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:07.110853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee12a52-f5da-4667-bf3a-b5e7eeaaa591 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:07.144190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e434ad2-33e0-4331-9c70-1e0bc9db6ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:07.144723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e434ad2-33e0-4331-9c70-1e0bc9db6ea7 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:07.144998 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e434ad2-33e0-4331-9c70-1e0bc9db6ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:07.145265 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e434ad2-33e0-4331-9c70-1e0bc9db6ea7 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:07.145945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e434ad2-33e0-4331-9c70-1e0bc9db6ea7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:07.146246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e434ad2-33e0-4331-9c70-1e0bc9db6ea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:07.147592 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e434ad2-33e0-4331-9c70-1e0bc9db6ea7 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:07.155661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 555/1102] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:34:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:07.163268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a887f-8add-45ba-b65b-d7e9dbe60491 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:07.220091 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:07.260660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a887f-8add-45ba-b65b-d7e9dbe60491 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:07.292550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68aa784-74f7-4be1-beba-4c6eff540ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:07.293163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68aa784-74f7-4be1-beba-4c6eff540ce2 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:07.293440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68aa784-74f7-4be1-beba-4c6eff540ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:07.293731 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68aa784-74f7-4be1-beba-4c6eff540ce2 None admin] PATH_INFO: `/v3/users/37b3d81d52434116be7c0238855bd243` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:07.294828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68aa784-74f7-4be1-beba-4c6eff540ce2 None admin] RBAC: Authorizing `identity:get_user(user_id=37b3d81d52434116be7c0238855bd243)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:07.296246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68aa784-74f7-4be1-beba-4c6eff540ce2 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:07.297809 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 548/1103] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:07 2022] GET /identity/v3/users/37b3d81d52434116be7c0238855bd243 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:07.304318 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e434ad2-33e0-4331-9c70-1e0bc9db6ea7 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:07.400335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e434ad2-33e0-4331-9c70-1e0bc9db6ea7 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:07.432874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e6420-fcf5-4c85-931c-0203469aaf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:07.433461 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e6420-fcf5-4c85-931c-0203469aaf2c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:07.433767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e6420-fcf5-4c85-931c-0203469aaf2c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:07.434042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e6420-fcf5-4c85-931c-0203469aaf2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:07.452418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:07.500418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e6420-fcf5-4c85-931c-0203469aaf2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:07.501899 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e6420-fcf5-4c85-931c-0203469aaf2c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:07.594704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 556/1104] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:34:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:34:07.601069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68aa784-74f7-4be1-beba-4c6eff540ce2 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:07.698409 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68aa784-74f7-4be1-beba-4c6eff540ce2 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:07.723965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:07.732408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bd33f-9e21-480c-bb16-9f7325f606de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:07.733028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bd33f-9e21-480c-bb16-9f7325f606de None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:07.733308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bd33f-9e21-480c-bb16-9f7325f606de None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:07.733578 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bd33f-9e21-480c-bb16-9f7325f606de None admin] PATH_INFO: `/v3/projects/c896707714b141faa1c51f1716574cb1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:07.734702 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9bd33f-9e21-480c-bb16-9f7325f606de None admin] RBAC: Authorizing `identity:get_project(project_id=c896707714b141faa1c51f1716574cb1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:07.736117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9bd33f-9e21-480c-bb16-9f7325f606de None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:07.737740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 549/1105] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:34:07 2022] GET /identity/v3/projects/c896707714b141faa1c51f1716574cb1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:07.743767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e6420-fcf5-4c85-931c-0203469aaf2c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:07.791503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:07.839346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e6420-fcf5-4c85-931c-0203469aaf2c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:07.872216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee5cdd-1170-4ee5-90a1-ca2445915d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:07.872793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee5cdd-1170-4ee5-90a1-ca2445915d4b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:07.873070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee5cdd-1170-4ee5-90a1-ca2445915d4b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:07.873341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee5cdd-1170-4ee5-90a1-ca2445915d4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:07.874071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee5cdd-1170-4ee5-90a1-ca2445915d4b None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=c896707714b141faa1c51f1716574cb1, user.id=37b3d81d52434116be7c0238855bd243) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:07.874373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee5cdd-1170-4ee5-90a1-ca2445915d4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:07.875776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee5cdd-1170-4ee5-90a1-ca2445915d4b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:07.881280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 557/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:34:07 2022] GET /identity/v3/role_assignments?user.id=37b3d81d52434116be7c0238855bd243&scope.project.id=c896707714b141faa1c51f1716574cb1&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:10.028010 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b57fa-5f28-476b-993f-43b99da40cf2 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:10.028532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b57fa-5f28-476b-993f-43b99da40cf2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:10.028642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b57fa-5f28-476b-993f-43b99da40cf2 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:10.029926 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 550/1107] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:10.035803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd99bca-aa4c-4c1e-b713-6dcd9a915466 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:10.036179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd99bca-aa4c-4c1e-b713-6dcd9a915466 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:10.036350 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd99bca-aa4c-4c1e-b713-6dcd9a915466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:10.053949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-3cd99bca-aa4c-4c1e-b713-6dcd9a915466 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:10.055113 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3cd99bca-aa4c-4c1e-b713-6dcd9a915466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:10.118160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 558/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:10.383726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b57fa-5f28-476b-993f-43b99da40cf2 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:10.480065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b57fa-5f28-476b-993f-43b99da40cf2 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:10.511752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b50a01-96eb-4b1b-8017-6b851ef01430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:10.512356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50a01-96eb-4b1b-8017-6b851ef01430 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:10.512634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50a01-96eb-4b1b-8017-6b851ef01430 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:10.512903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50a01-96eb-4b1b-8017-6b851ef01430 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:10.514010 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b50a01-96eb-4b1b-8017-6b851ef01430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:10.515423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b50a01-96eb-4b1b-8017-6b851ef01430 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:10.516937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 551/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:34:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:10.522577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd99bca-aa4c-4c1e-b713-6dcd9a915466 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:10.565999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:10.618130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd99bca-aa4c-4c1e-b713-6dcd9a915466 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:10.650425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed600f49-608f-458a-8f0e-fa354a56e3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:10.650968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed600f49-608f-458a-8f0e-fa354a56e3dc None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:10.651245 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed600f49-608f-458a-8f0e-fa354a56e3dc None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:10.651514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed600f49-608f-458a-8f0e-fa354a56e3dc None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:10.652163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed600f49-608f-458a-8f0e-fa354a56e3dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:10.654072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed600f49-608f-458a-8f0e-fa354a56e3dc None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:10.675411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 559/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:34:10 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:34:12.830238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c34de-2388-4e09-b2ca-de83a1cf6596 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:12.830756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c34de-2388-4e09-b2ca-de83a1cf6596 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:12.830756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c34de-2388-4e09-b2ca-de83a1cf6596 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:12.832093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 552/1111] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:12.837239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35430c-f9a8-4136-bc18-349485ef8136 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:12.837626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35430c-f9a8-4136-bc18-349485ef8136 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:12.837842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35430c-f9a8-4136-bc18-349485ef8136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:12.857113 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-af35430c-f9a8-4136-bc18-349485ef8136 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:12.858330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-af35430c-f9a8-4136-bc18-349485ef8136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:12.923222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 560/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:13.190729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c34de-2388-4e09-b2ca-de83a1cf6596 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:13.252808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:13.293173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c34de-2388-4e09-b2ca-de83a1cf6596 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:13.324938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a9bec-90fa-4e66-88c4-b620ab2a1733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:13.325553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a9bec-90fa-4e66-88c4-b620ab2a1733 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:13.325860 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a9bec-90fa-4e66-88c4-b620ab2a1733 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:13.326131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a9bec-90fa-4e66-88c4-b620ab2a1733 None admin] PATH_INFO: `/v3/roles/c0bf52b447564f769d707e96facb20a1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:13.327501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a9bec-90fa-4e66-88c4-b620ab2a1733 None admin] RBAC: Authorizing `identity:get_role(role_id=c0bf52b447564f769d707e96facb20a1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:13.328854 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a9bec-90fa-4e66-88c4-b620ab2a1733 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:13.330191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 553/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:13 2022] GET /identity/v3/roles/c0bf52b447564f769d707e96facb20a1 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:13.336577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35430c-f9a8-4136-bc18-349485ef8136 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:13.365116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:13.433562 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35430c-f9a8-4136-bc18-349485ef8136 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:13.466831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebe8d3-6fa5-4a11-abc0-22a148113c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:13.467389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebe8d3-6fa5-4a11-abc0-22a148113c9d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:13.467674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebe8d3-6fa5-4a11-abc0-22a148113c9d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:13.467947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebe8d3-6fa5-4a11-abc0-22a148113c9d None admin] PATH_INFO: `/v3/users/46b38727b3b54cb7bfa6ffa5ba0c999a` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:13.480873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebe8d3-6fa5-4a11-abc0-22a148113c9d None admin] RBAC: Authorizing `identity:get_user(user_id=46b38727b3b54cb7bfa6ffa5ba0c999a)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:13.482332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebe8d3-6fa5-4a11-abc0-22a148113c9d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:13.483889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 561/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:13 2022] GET /identity/v3/users/46b38727b3b54cb7bfa6ffa5ba0c999a => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:13.489682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a9bec-90fa-4e66-88c4-b620ab2a1733 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:13.584539 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a9bec-90fa-4e66-88c4-b620ab2a1733 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:13.612268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:13.617528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ea683-708d-4346-abac-59c5fdb0f567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:13.618143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ea683-708d-4346-abac-59c5fdb0f567 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:13.618423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ea683-708d-4346-abac-59c5fdb0f567 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:13.618696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ea683-708d-4346-abac-59c5fdb0f567 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:13.687914 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ea683-708d-4346-abac-59c5fdb0f567 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:13.689354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ea683-708d-4346-abac-59c5fdb0f567 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:13.783500 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 554/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:34:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:34:13.789533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebe8d3-6fa5-4a11-abc0-22a148113c9d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:13.873297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:13.884899 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebe8d3-6fa5-4a11-abc0-22a148113c9d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:13.917486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d5b8df-8cb7-4dbc-bbfc-2941a9843737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:13.918125 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5b8df-8cb7-4dbc-bbfc-2941a9843737 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:13.918401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5b8df-8cb7-4dbc-bbfc-2941a9843737 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:13.918670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d5b8df-8cb7-4dbc-bbfc-2941a9843737 None admin] PATH_INFO: `/v3/projects/c896707714b141faa1c51f1716574cb1` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:13.919765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5b8df-8cb7-4dbc-bbfc-2941a9843737 None admin] RBAC: Authorizing `identity:get_project(project_id=c896707714b141faa1c51f1716574cb1)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:13.921187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d5b8df-8cb7-4dbc-bbfc-2941a9843737 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:13.922793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 562/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:34:13 2022] GET /identity/v3/projects/c896707714b141faa1c51f1716574cb1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:13.928803 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ea683-708d-4346-abac-59c5fdb0f567 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:13.957793 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:14.024072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ea683-708d-4346-abac-59c5fdb0f567 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:14.056204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88089bf9-74f4-4c30-973e-4448d66e7a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:14.056732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88089bf9-74f4-4c30-973e-4448d66e7a89 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:14.057006 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88089bf9-74f4-4c30-973e-4448d66e7a89 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:14.057272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88089bf9-74f4-4c30-973e-4448d66e7a89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:14.058008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88089bf9-74f4-4c30-973e-4448d66e7a89 None admin] RBAC: Adding query filter params (role.id=c0bf52b447564f769d707e96facb20a1, scope.project.id=c896707714b141faa1c51f1716574cb1, user.id=46b38727b3b54cb7bfa6ffa5ba0c999a) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:14.058310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88089bf9-74f4-4c30-973e-4448d66e7a89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:14.059710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88089bf9-74f4-4c30-973e-4448d66e7a89 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:14.064790 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 555/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:34:13 2022] GET /identity/v3/role_assignments?user.id=46b38727b3b54cb7bfa6ffa5ba0c999a&scope.project.id=c896707714b141faa1c51f1716574cb1&role.id=c0bf52b447564f769d707e96facb20a1 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:16.126733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12627e2b-ce5e-4638-a413-b44205098bc6 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:16.127292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12627e2b-ce5e-4638-a413-b44205098bc6 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:16.127292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12627e2b-ce5e-4638-a413-b44205098bc6 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:16.129224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 563/1118] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:16.134237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c52f1-72ea-4ac6-a656-f9537194ca72 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:16.134609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c52f1-72ea-4ac6-a656-f9537194ca72 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:16.134784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c52f1-72ea-4ac6-a656-f9537194ca72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:16.152375 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c80c52f1-72ea-4ac6-a656-f9537194ca72 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:16.153538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c80c52f1-72ea-4ac6-a656-f9537194ca72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:16.215736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 556/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:16.483853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12627e2b-ce5e-4638-a413-b44205098bc6 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:16.581824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12627e2b-ce5e-4638-a413-b44205098bc6 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:16.615075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cf79b-8d1a-42f8-ab16-61f6fb8bd3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:16.615683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cf79b-8d1a-42f8-ab16-61f6fb8bd3a1 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:16.615967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cf79b-8d1a-42f8-ab16-61f6fb8bd3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:16.616238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cf79b-8d1a-42f8-ab16-61f6fb8bd3a1 None admin] PATH_INFO: `/v3/roles/c0bf52b447564f769d707e96facb20a1` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:16.617600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cf79b-8d1a-42f8-ab16-61f6fb8bd3a1 None admin] RBAC: Authorizing `identity:get_role(role_id=c0bf52b447564f769d707e96facb20a1)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:16.618979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cf79b-8d1a-42f8-ab16-61f6fb8bd3a1 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:16.620251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 564/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:16 2022] GET /identity/v3/roles/c0bf52b447564f769d707e96facb20a1 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:16.625806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c52f1-72ea-4ac6-a656-f9537194ca72 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:16.651384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:16.723499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c52f1-72ea-4ac6-a656-f9537194ca72 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:16.755517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90feb5-6595-41a6-884a-216da79d465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:16.756075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90feb5-6595-41a6-884a-216da79d465a None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:16.756353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90feb5-6595-41a6-884a-216da79d465a None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:16.756625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90feb5-6595-41a6-884a-216da79d465a None admin] PATH_INFO: `/v3/users/46b38727b3b54cb7bfa6ffa5ba0c999a` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:16.757741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90feb5-6595-41a6-884a-216da79d465a None admin] RBAC: Authorizing `identity:get_user(user_id=46b38727b3b54cb7bfa6ffa5ba0c999a)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:16.759170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90feb5-6595-41a6-884a-216da79d465a None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:16.760733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 557/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:16 2022] GET /identity/v3/users/46b38727b3b54cb7bfa6ffa5ba0c999a => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:16.766600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cf79b-8d1a-42f8-ab16-61f6fb8bd3a1 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:16.792244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:16.864575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cf79b-8d1a-42f8-ab16-61f6fb8bd3a1 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:16.896620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76be6e-23e1-4ee9-a109-938e3d0d3166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:16.897225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76be6e-23e1-4ee9-a109-938e3d0d3166 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:16.897506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76be6e-23e1-4ee9-a109-938e3d0d3166 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:16.897807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76be6e-23e1-4ee9-a109-938e3d0d3166 None admin] PATH_INFO: `/v3/projects/c896707714b141faa1c51f1716574cb1` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:16.898913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76be6e-23e1-4ee9-a109-938e3d0d3166 None admin] RBAC: Authorizing `identity:get_project(project_id=c896707714b141faa1c51f1716574cb1)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:16.900350 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76be6e-23e1-4ee9-a109-938e3d0d3166 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:16.902053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 565/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:34:16 2022] GET /identity/v3/projects/c896707714b141faa1c51f1716574cb1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:16.908037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90feb5-6595-41a6-884a-216da79d465a None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:16.996784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:17.003402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90feb5-6595-41a6-884a-216da79d465a None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:17.036095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e39d62-d1d2-493d-8f78-eb017f1e0254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:17.036729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e39d62-d1d2-493d-8f78-eb017f1e0254 None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:17.037006 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e39d62-d1d2-493d-8f78-eb017f1e0254 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:17.037276 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e39d62-d1d2-493d-8f78-eb017f1e0254 None admin] PATH_INFO: `/v3/projects/c896707714b141faa1c51f1716574cb1/users/46b38727b3b54cb7bfa6ffa5ba0c999a/roles/c0bf52b447564f769d707e96facb20a1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:17.039396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e39d62-d1d2-493d-8f78-eb017f1e0254 None admin] RBAC: Authorizing `identity:create_grant(project_id=c896707714b141faa1c51f1716574cb1, user_id=46b38727b3b54cb7bfa6ffa5ba0c999a, role_id=c0bf52b447564f769d707e96facb20a1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:17.040817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e39d62-d1d2-493d-8f78-eb017f1e0254 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:17.052616 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 558/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:34:16 2022] PUT /identity/v3/projects/c896707714b141faa1c51f1716574cb1/users/46b38727b3b54cb7bfa6ffa5ba0c999a/roles/c0bf52b447564f769d707e96facb20a1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:34:19.185018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edd7b0-1580-4b46-be1d-302736aa190f None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:19.185567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edd7b0-1580-4b46-be1d-302736aa190f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:19.185567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edd7b0-1580-4b46-be1d-302736aa190f None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:19.187786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 566/1124] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:19.193742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40fe91-b31d-4c81-ae19-4b8617810585 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:19.194187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40fe91-b31d-4c81-ae19-4b8617810585 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:19.194287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40fe91-b31d-4c81-ae19-4b8617810585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:19.212406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-8a40fe91-b31d-4c81-ae19-4b8617810585 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:19.213573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8a40fe91-b31d-4c81-ae19-4b8617810585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:19.276756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 559/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:19.545266 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edd7b0-1580-4b46-be1d-302736aa190f None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:19.634981 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:19.645618 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edd7b0-1580-4b46-be1d-302736aa190f None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:19.678758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625f371-4671-4446-855c-b0691a13a74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:19.679382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f371-4671-4446-855c-b0691a13a74c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:19.679662 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f371-4671-4446-855c-b0691a13a74c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:19.679934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625f371-4671-4446-855c-b0691a13a74c None admin] PATH_INFO: `/v3/roles/c0bf52b447564f769d707e96facb20a1` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:19.681297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625f371-4671-4446-855c-b0691a13a74c None admin] RBAC: Authorizing `identity:get_role(role_id=c0bf52b447564f769d707e96facb20a1)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:19.682710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625f371-4671-4446-855c-b0691a13a74c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:19.684016 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 567/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:19 2022] GET /identity/v3/roles/c0bf52b447564f769d707e96facb20a1 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:19.689875 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40fe91-b31d-4c81-ae19-4b8617810585 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:19.776061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:19.792421 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40fe91-b31d-4c81-ae19-4b8617810585 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:19.824614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca9fca-990f-4d4e-b629-ea8b2466562e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:19.825168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca9fca-990f-4d4e-b629-ea8b2466562e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:19.825440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca9fca-990f-4d4e-b629-ea8b2466562e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:19.825725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca9fca-990f-4d4e-b629-ea8b2466562e None admin] PATH_INFO: `/v3/users/46b38727b3b54cb7bfa6ffa5ba0c999a` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:19.826821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca9fca-990f-4d4e-b629-ea8b2466562e None admin] RBAC: Authorizing `identity:get_user(user_id=46b38727b3b54cb7bfa6ffa5ba0c999a)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:19.828235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca9fca-990f-4d4e-b629-ea8b2466562e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:19.829794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 560/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:19 2022] GET /identity/v3/users/46b38727b3b54cb7bfa6ffa5ba0c999a => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:19.836174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625f371-4671-4446-855c-b0691a13a74c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:19.932731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625f371-4671-4446-855c-b0691a13a74c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:19.964702 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afae28-8bd5-4e36-ad37-4f81a9701c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:19.965284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afae28-8bd5-4e36-ad37-4f81a9701c80 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:19.965560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afae28-8bd5-4e36-ad37-4f81a9701c80 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:19.965866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afae28-8bd5-4e36-ad37-4f81a9701c80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:19.992436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:20.030540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afae28-8bd5-4e36-ad37-4f81a9701c80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:20.031973 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afae28-8bd5-4e36-ad37-4f81a9701c80 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:20.124640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 568/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:34:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:34:20.130992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca9fca-990f-4d4e-b629-ea8b2466562e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:20.224613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca9fca-990f-4d4e-b629-ea8b2466562e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:20.256611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c601a-7430-47a2-9dfc-0d5701c7d95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:20.257213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c601a-7430-47a2-9dfc-0d5701c7d95c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:20.257488 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c601a-7430-47a2-9dfc-0d5701c7d95c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:20.257779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c601a-7430-47a2-9dfc-0d5701c7d95c None admin] PATH_INFO: `/v3/projects/c896707714b141faa1c51f1716574cb1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:20.258863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c601a-7430-47a2-9dfc-0d5701c7d95c None admin] RBAC: Authorizing `identity:get_project(project_id=c896707714b141faa1c51f1716574cb1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:20.260261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c601a-7430-47a2-9dfc-0d5701c7d95c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:20.261835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 561/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:34:20 2022] GET /identity/v3/projects/c896707714b141faa1c51f1716574cb1 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:20.267738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afae28-8bd5-4e36-ad37-4f81a9701c80 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:20.324226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:20.363779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afae28-8bd5-4e36-ad37-4f81a9701c80 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:20.396341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cb961-33ac-4ec7-b02b-9d448c33de95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:20.396875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cb961-33ac-4ec7-b02b-9d448c33de95 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:20.397155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cb961-33ac-4ec7-b02b-9d448c33de95 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:20.397427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cb961-33ac-4ec7-b02b-9d448c33de95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:20.398166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32cb961-33ac-4ec7-b02b-9d448c33de95 None admin] RBAC: Adding query filter params (role.id=c0bf52b447564f769d707e96facb20a1, scope.project.id=c896707714b141faa1c51f1716574cb1, user.id=46b38727b3b54cb7bfa6ffa5ba0c999a) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:20.398474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32cb961-33ac-4ec7-b02b-9d448c33de95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:20.399886 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32cb961-33ac-4ec7-b02b-9d448c33de95 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:20.405417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 569/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:34:20 2022] GET /identity/v3/role_assignments?user.id=46b38727b3b54cb7bfa6ffa5ba0c999a&scope.project.id=c896707714b141faa1c51f1716574cb1&role.id=c0bf52b447564f769d707e96facb20a1 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:22.523648 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e2898-bf04-4338-91a2-e9dcd4cb7611 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:22.524181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e2898-bf04-4338-91a2-e9dcd4cb7611 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:22.524181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e2898-bf04-4338-91a2-e9dcd4cb7611 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:22.525505 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 562/1131] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:22.530631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1660a-c361-4df1-a6e5-d322a9a3360d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:22.531024 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1660a-c361-4df1-a6e5-d322a9a3360d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:22.531185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1660a-c361-4df1-a6e5-d322a9a3360d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:22.549312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-2cf1660a-c361-4df1-a6e5-d322a9a3360d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:22.551081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2cf1660a-c361-4df1-a6e5-d322a9a3360d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:22.615102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 570/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:22.879776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e2898-bf04-4338-91a2-e9dcd4cb7611 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:22.901069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:22.977222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e2898-bf04-4338-91a2-e9dcd4cb7611 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:23.008999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14687e-fa01-4709-a0ff-3dcb6fc04789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:23.009607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14687e-fa01-4709-a0ff-3dcb6fc04789 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:23.009903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14687e-fa01-4709-a0ff-3dcb6fc04789 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:23.010171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14687e-fa01-4709-a0ff-3dcb6fc04789 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:23.011252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14687e-fa01-4709-a0ff-3dcb6fc04789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:23.012664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14687e-fa01-4709-a0ff-3dcb6fc04789 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:23.014206 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 563/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:34:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:23.020333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1660a-c361-4df1-a6e5-d322a9a3360d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:23.073028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:23.116359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1660a-c361-4df1-a6e5-d322a9a3360d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:23.150154 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34a864-b84e-4b2f-8f51-289043266fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:23.150684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34a864-b84e-4b2f-8f51-289043266fc7 None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:23.150961 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34a864-b84e-4b2f-8f51-289043266fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:23.151231 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34a864-b84e-4b2f-8f51-289043266fc7 None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:23.151880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34a864-b84e-4b2f-8f51-289043266fc7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:23.153289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34a864-b84e-4b2f-8f51-289043266fc7 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:23.178697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 571/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:34:23 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:34:25.334374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdcb72-83ee-4091-9741-0259f63f2345 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:25.334923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdcb72-83ee-4091-9741-0259f63f2345 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:25.334923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdcb72-83ee-4091-9741-0259f63f2345 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:25.336253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 564/1135] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:25.341304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbe064-6e8c-4904-9d90-2e7f6e123e31 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:25.341744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbe064-6e8c-4904-9d90-2e7f6e123e31 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:25.341876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbe064-6e8c-4904-9d90-2e7f6e123e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:25.360398 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6afbe064-6e8c-4904-9d90-2e7f6e123e31 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:25.361567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6afbe064-6e8c-4904-9d90-2e7f6e123e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:25.425005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 572/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:25.694132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cdcb72-83ee-4091-9741-0259f63f2345 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:25.778218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:25.790873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cdcb72-83ee-4091-9741-0259f63f2345 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:25.824034 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9664d52-fb6f-4a8b-be4c-6747f38434ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:25.824655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9664d52-fb6f-4a8b-be4c-6747f38434ca None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:25.824932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9664d52-fb6f-4a8b-be4c-6747f38434ca None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:25.825201 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9664d52-fb6f-4a8b-be4c-6747f38434ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:25.826293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9664d52-fb6f-4a8b-be4c-6747f38434ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:25.827715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9664d52-fb6f-4a8b-be4c-6747f38434ca None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:25.829250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 565/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:34:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:25.835040 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbe064-6e8c-4904-9d90-2e7f6e123e31 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:25.868380 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:25.931555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbe064-6e8c-4904-9d90-2e7f6e123e31 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:25.963937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f9628-5ae5-4ee7-b7dd-2d45df0a21aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:25.964524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f9628-5ae5-4ee7-b7dd-2d45df0a21aa None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:25.964801 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f9628-5ae5-4ee7-b7dd-2d45df0a21aa None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:25.965071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f9628-5ae5-4ee7-b7dd-2d45df0a21aa None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:25.965731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f9628-5ae5-4ee7-b7dd-2d45df0a21aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:25.967143 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668f9628-5ae5-4ee7-b7dd-2d45df0a21aa None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:25.988120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 573/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:34:25 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:34:28.124924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b408f9-ba98-4a5e-b15e-f674279b6357 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:28.125452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b408f9-ba98-4a5e-b15e-f674279b6357 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:28.125452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b408f9-ba98-4a5e-b15e-f674279b6357 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:28.126836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 566/1139] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:28.132504 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b94ae1-a593-4d2e-812b-c85505a0a81e None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:28.132900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b94ae1-a593-4d2e-812b-c85505a0a81e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:28.133040 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b94ae1-a593-4d2e-812b-c85505a0a81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:28.150668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-06b94ae1-a593-4d2e-812b-c85505a0a81e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:28.151838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-06b94ae1-a593-4d2e-812b-c85505a0a81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:28.215004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 574/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:28.480562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b408f9-ba98-4a5e-b15e-f674279b6357 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:28.577492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b408f9-ba98-4a5e-b15e-f674279b6357 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:28.609732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c861e47-5070-42f6-9173-db577437501e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:28.610345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861e47-5070-42f6-9173-db577437501e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:28.610624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861e47-5070-42f6-9173-db577437501e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:28.610894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861e47-5070-42f6-9173-db577437501e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:28.620620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c861e47-5070-42f6-9173-db577437501e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-0c861e47-5070-42f6-9173-db577437501e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:34:28.625302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:28.626490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:34:28.628209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 567/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:34:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:34:28.632641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b94ae1-a593-4d2e-812b-c85505a0a81e None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:28.656137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:28.730738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b94ae1-a593-4d2e-812b-c85505a0a81e None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:28.763977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e22275-0f66-41e0-9d65-15af3ff7f859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:28.764572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e22275-0f66-41e0-9d65-15af3ff7f859 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:28.764851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e22275-0f66-41e0-9d65-15af3ff7f859 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:28.765121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e22275-0f66-41e0-9d65-15af3ff7f859 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:28.765800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e22275-0f66-41e0-9d65-15af3ff7f859 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:28.766101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e22275-0f66-41e0-9d65-15af3ff7f859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:28.767459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e22275-0f66-41e0-9d65-15af3ff7f859 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:28.775117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 575/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:34:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:28.781069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c861e47-5070-42f6-9173-db577437501e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:28.799538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:28.879167 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c861e47-5070-42f6-9173-db577437501e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:28.912250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27e1f1-5149-41e6-8cbc-b0ff3caf4377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:28.912866 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27e1f1-5149-41e6-8cbc-b0ff3caf4377 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:28.913143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27e1f1-5149-41e6-8cbc-b0ff3caf4377 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:28.913414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27e1f1-5149-41e6-8cbc-b0ff3caf4377 None admin] PATH_INFO: `/v3/users/62f55e6f79724fd1b6ba48183ea35a75` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:28.927041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27e1f1-5149-41e6-8cbc-b0ff3caf4377 None admin] RBAC: Authorizing `identity:get_user(user_id=62f55e6f79724fd1b6ba48183ea35a75)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:28.928470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27e1f1-5149-41e6-8cbc-b0ff3caf4377 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:28.930064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 568/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:28 2022] GET /identity/v3/users/62f55e6f79724fd1b6ba48183ea35a75 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:28.935911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e22275-0f66-41e0-9d65-15af3ff7f859 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:29.018270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:29.034460 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e22275-0f66-41e0-9d65-15af3ff7f859 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:29.066834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5c760-f627-43d0-858c-9e0fdbf7d508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:29.067420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5c760-f627-43d0-858c-9e0fdbf7d508 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:29.067700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5c760-f627-43d0-858c-9e0fdbf7d508 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:29.067974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5c760-f627-43d0-858c-9e0fdbf7d508 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:29.132444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5c760-f627-43d0-858c-9e0fdbf7d508 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:29.133905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5c760-f627-43d0-858c-9e0fdbf7d508 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:29.212896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:29.229651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 576/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:34:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:34:29.235724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27e1f1-5149-41e6-8cbc-b0ff3caf4377 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:29.306938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:29.331707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27e1f1-5149-41e6-8cbc-b0ff3caf4377 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:29.365120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf8a6a-89f0-4400-a5b7-871e79dc807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:29.365760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8a6a-89f0-4400-a5b7-871e79dc807f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:29.366036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8a6a-89f0-4400-a5b7-871e79dc807f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:29.366303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf8a6a-89f0-4400-a5b7-871e79dc807f None admin] PATH_INFO: `/v3/projects/45b9870f625248f2b1dc478f78fc22a9` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:29.367382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf8a6a-89f0-4400-a5b7-871e79dc807f None admin] RBAC: Authorizing `identity:get_project(project_id=45b9870f625248f2b1dc478f78fc22a9)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:29.368796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf8a6a-89f0-4400-a5b7-871e79dc807f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:29.370427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 569/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:34:29 2022] GET /identity/v3/projects/45b9870f625248f2b1dc478f78fc22a9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:29.376597 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5c760-f627-43d0-858c-9e0fdbf7d508 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:29.473116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5c760-f627-43d0-858c-9e0fdbf7d508 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:29.506270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75e7a1-9ca0-4f01-9904-6a7589210b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:29.506867 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75e7a1-9ca0-4f01-9904-6a7589210b69 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:29.507145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75e7a1-9ca0-4f01-9904-6a7589210b69 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:29.507418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75e7a1-9ca0-4f01-9904-6a7589210b69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:29.508142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75e7a1-9ca0-4f01-9904-6a7589210b69 None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=45b9870f625248f2b1dc478f78fc22a9, user.id=62f55e6f79724fd1b6ba48183ea35a75) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:29.508451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75e7a1-9ca0-4f01-9904-6a7589210b69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:29.509901 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75e7a1-9ca0-4f01-9904-6a7589210b69 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:29.515096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 577/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:34:29 2022] GET /identity/v3/role_assignments?user.id=62f55e6f79724fd1b6ba48183ea35a75&scope.project.id=45b9870f625248f2b1dc478f78fc22a9&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:31.612760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef020a5d-74d6-4e10-b059-82dbc9c812c5 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:31.613328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef020a5d-74d6-4e10-b059-82dbc9c812c5 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:31.613328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef020a5d-74d6-4e10-b059-82dbc9c812c5 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:31.614655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 570/1147] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:31.619960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d330397-8bf6-46b8-b51e-3ab1fa891487 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:31.620390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d330397-8bf6-46b8-b51e-3ab1fa891487 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:31.620518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d330397-8bf6-46b8-b51e-3ab1fa891487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:31.638184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-7d330397-8bf6-46b8-b51e-3ab1fa891487 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:31.639361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7d330397-8bf6-46b8-b51e-3ab1fa891487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:31.682226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:31.704670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 578/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:31.974154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef020a5d-74d6-4e10-b059-82dbc9c812c5 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:32.072346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef020a5d-74d6-4e10-b059-82dbc9c812c5 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:32.104910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cec07-c8c1-434c-9176-ce4ea6a44ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:32.105532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cec07-c8c1-434c-9176-ce4ea6a44ed7 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:32.105849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cec07-c8c1-434c-9176-ce4ea6a44ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:32.106118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cec07-c8c1-434c-9176-ce4ea6a44ed7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:32.116388 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cec07-c8c1-434c-9176-ce4ea6a44ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-c52cec07-c8c1-434c-9176-ce4ea6a44ed7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:34:32.121026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:32.122218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:34:32.123362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 571/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:34:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:34:32.128199 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d330397-8bf6-46b8-b51e-3ab1fa891487 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:32.222905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d330397-8bf6-46b8-b51e-3ab1fa891487 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:32.256214 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fb072-11eb-4b43-87eb-911f14550fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:32.256769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fb072-11eb-4b43-87eb-911f14550fed None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:32.257055 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fb072-11eb-4b43-87eb-911f14550fed None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:32.257329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fb072-11eb-4b43-87eb-911f14550fed None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:32.258026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fb072-11eb-4b43-87eb-911f14550fed None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:32.258336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fb072-11eb-4b43-87eb-911f14550fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:32.259701 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fb072-11eb-4b43-87eb-911f14550fed None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:32.267593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 579/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:34:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:32.273280 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cec07-c8c1-434c-9176-ce4ea6a44ed7 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:32.276634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:32.373124 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cec07-c8c1-434c-9176-ce4ea6a44ed7 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:32.405428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703c247-bbd4-4b11-9fa8-defa2e9d5ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:32.406072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703c247-bbd4-4b11-9fa8-defa2e9d5ddb None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:32.406352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703c247-bbd4-4b11-9fa8-defa2e9d5ddb None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:32.406623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703c247-bbd4-4b11-9fa8-defa2e9d5ddb None admin] PATH_INFO: `/v3/users/62f55e6f79724fd1b6ba48183ea35a75` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:32.407728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703c247-bbd4-4b11-9fa8-defa2e9d5ddb None admin] RBAC: Authorizing `identity:get_user(user_id=62f55e6f79724fd1b6ba48183ea35a75)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:32.409725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703c247-bbd4-4b11-9fa8-defa2e9d5ddb None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:32.411289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 572/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:32 2022] GET /identity/v3/users/62f55e6f79724fd1b6ba48183ea35a75 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:32.417123 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fb072-11eb-4b43-87eb-911f14550fed None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:32.457563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:32.514338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fb072-11eb-4b43-87eb-911f14550fed None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:32.547292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc4b08-bb1c-4c52-9958-ebac63e85491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:32.547907 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc4b08-bb1c-4c52-9958-ebac63e85491 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:32.548185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc4b08-bb1c-4c52-9958-ebac63e85491 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:32.548457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc4b08-bb1c-4c52-9958-ebac63e85491 None admin] PATH_INFO: `/v3/projects/45b9870f625248f2b1dc478f78fc22a9` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:32.549562 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc4b08-bb1c-4c52-9958-ebac63e85491 None admin] RBAC: Authorizing `identity:get_project(project_id=45b9870f625248f2b1dc478f78fc22a9)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:32.551023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc4b08-bb1c-4c52-9958-ebac63e85491 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:32.552613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 580/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:34:32 2022] GET /identity/v3/projects/45b9870f625248f2b1dc478f78fc22a9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:32.558592 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703c247-bbd4-4b11-9fa8-defa2e9d5ddb None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:32.625408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:32.660122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703c247-bbd4-4b11-9fa8-defa2e9d5ddb None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:32.701246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508db353-1b08-4605-aac4-88b68ca83b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:32.701931 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508db353-1b08-4605-aac4-88b68ca83b64 None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:32.702216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508db353-1b08-4605-aac4-88b68ca83b64 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:32.702493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508db353-1b08-4605-aac4-88b68ca83b64 None admin] PATH_INFO: `/v3/projects/45b9870f625248f2b1dc478f78fc22a9/users/62f55e6f79724fd1b6ba48183ea35a75/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:32.704384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508db353-1b08-4605-aac4-88b68ca83b64 None admin] RBAC: Authorizing `identity:create_grant(project_id=45b9870f625248f2b1dc478f78fc22a9, user_id=62f55e6f79724fd1b6ba48183ea35a75, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:32.706386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508db353-1b08-4605-aac4-88b68ca83b64 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:32.722807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 573/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:34:32 2022] PUT /identity/v3/projects/45b9870f625248f2b1dc478f78fc22a9/users/62f55e6f79724fd1b6ba48183ea35a75/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:34:34.838334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc04c6a-c326-4b97-a17d-a821731661e9 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:34.838890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc04c6a-c326-4b97-a17d-a821731661e9 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:34.838890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc04c6a-c326-4b97-a17d-a821731661e9 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:34.840212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 581/1154] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:34.845418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad47c4-67f7-4612-b13b-efdfe23bdaaf None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:34.846451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad47c4-67f7-4612-b13b-efdfe23bdaaf None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:34.846599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad47c4-67f7-4612-b13b-efdfe23bdaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:34.864653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-56ad47c4-67f7-4612-b13b-efdfe23bdaaf None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:34.865866 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-56ad47c4-67f7-4612-b13b-efdfe23bdaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:34.929329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 574/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:35.201449 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc04c6a-c326-4b97-a17d-a821731661e9 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:35.300462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc04c6a-c326-4b97-a17d-a821731661e9 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:35.310007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:35.334988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4ea60-b909-4eba-9f2b-827919521afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:35.335596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4ea60-b909-4eba-9f2b-827919521afa None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:35.335871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4ea60-b909-4eba-9f2b-827919521afa None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:35.336141 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4ea60-b909-4eba-9f2b-827919521afa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:35.345727 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4ea60-b909-4eba-9f2b-827919521afa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-98d4ea60-b909-4eba-9f2b-827919521afa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:34:35.350373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:35.353766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:34:35.354885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 582/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:34:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:34:35.357774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad47c4-67f7-4612-b13b-efdfe23bdaaf None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:35.414349 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:35.454044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad47c4-67f7-4612-b13b-efdfe23bdaaf None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:35.485830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d665ec9-4bcf-4f8c-a5a0-66fdc132c2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:35.486966 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d665ec9-4bcf-4f8c-a5a0-66fdc132c2a0 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:35.487243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d665ec9-4bcf-4f8c-a5a0-66fdc132c2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:35.487512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d665ec9-4bcf-4f8c-a5a0-66fdc132c2a0 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:35.488158 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d665ec9-4bcf-4f8c-a5a0-66fdc132c2a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:35.488455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d665ec9-4bcf-4f8c-a5a0-66fdc132c2a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:35.489831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d665ec9-4bcf-4f8c-a5a0-66fdc132c2a0 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:35.497442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 575/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:34:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:35.504103 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4ea60-b909-4eba-9f2b-827919521afa None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:35.600384 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4ea60-b909-4eba-9f2b-827919521afa None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:35.632693 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd154a9-654c-479f-bd46-e64f8752c01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:35.633297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd154a9-654c-479f-bd46-e64f8752c01c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:35.633576 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd154a9-654c-479f-bd46-e64f8752c01c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:35.633880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd154a9-654c-479f-bd46-e64f8752c01c None admin] PATH_INFO: `/v3/users/62f55e6f79724fd1b6ba48183ea35a75` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:35.634990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd154a9-654c-479f-bd46-e64f8752c01c None admin] RBAC: Authorizing `identity:get_user(user_id=62f55e6f79724fd1b6ba48183ea35a75)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:35.636409 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd154a9-654c-479f-bd46-e64f8752c01c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:35.637972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 583/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:35 2022] GET /identity/v3/users/62f55e6f79724fd1b6ba48183ea35a75 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:35.644130 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d665ec9-4bcf-4f8c-a5a0-66fdc132c2a0 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:35.739714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d665ec9-4bcf-4f8c-a5a0-66fdc132c2a0 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:35.760373 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:35.771926 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee3ec6-872f-4248-8f07-e3941bb492e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:35.772517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee3ec6-872f-4248-8f07-e3941bb492e6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:35.772793 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee3ec6-872f-4248-8f07-e3941bb492e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:35.773064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee3ec6-872f-4248-8f07-e3941bb492e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:35.835838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee3ec6-872f-4248-8f07-e3941bb492e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:35.837258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee3ec6-872f-4248-8f07-e3941bb492e6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:35.930420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 576/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:34:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:34:35.936298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd154a9-654c-479f-bd46-e64f8752c01c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:35.971310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:36.032817 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd154a9-654c-479f-bd46-e64f8752c01c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:36.065384 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8058e1-ebd3-4d6f-afc4-42cd1b47a124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:36.066020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8058e1-ebd3-4d6f-afc4-42cd1b47a124 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:36.066297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8058e1-ebd3-4d6f-afc4-42cd1b47a124 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:36.066563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8058e1-ebd3-4d6f-afc4-42cd1b47a124 None admin] PATH_INFO: `/v3/projects/45b9870f625248f2b1dc478f78fc22a9` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:36.067689 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8058e1-ebd3-4d6f-afc4-42cd1b47a124 None admin] RBAC: Authorizing `identity:get_project(project_id=45b9870f625248f2b1dc478f78fc22a9)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:36.069130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8058e1-ebd3-4d6f-afc4-42cd1b47a124 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:36.070779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 584/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:34:35 2022] GET /identity/v3/projects/45b9870f625248f2b1dc478f78fc22a9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:36.076856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee3ec6-872f-4248-8f07-e3941bb492e6 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:36.100342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:36.171249 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee3ec6-872f-4248-8f07-e3941bb492e6 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:36.203542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4931cb9-2759-4e9d-a3d7-4711b31bce2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:36.204126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4931cb9-2759-4e9d-a3d7-4711b31bce2d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:36.204398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4931cb9-2759-4e9d-a3d7-4711b31bce2d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:36.204663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4931cb9-2759-4e9d-a3d7-4711b31bce2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:36.205396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4931cb9-2759-4e9d-a3d7-4711b31bce2d None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=45b9870f625248f2b1dc478f78fc22a9, user.id=62f55e6f79724fd1b6ba48183ea35a75) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:36.205722 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4931cb9-2759-4e9d-a3d7-4711b31bce2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:36.207126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4931cb9-2759-4e9d-a3d7-4711b31bce2d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:36.212548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 577/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:34:36 2022] GET /identity/v3/role_assignments?user.id=62f55e6f79724fd1b6ba48183ea35a75&scope.project.id=45b9870f625248f2b1dc478f78fc22a9&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:38.325191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c4b01-615b-444d-b872-fa2642778183 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:38.325849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c4b01-615b-444d-b872-fa2642778183 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:38.325849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c4b01-615b-444d-b872-fa2642778183 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:38.327117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 585/1162] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:38.332359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bf8ec-59e5-42c1-972a-cdbee6c1e9ff None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:38.332776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bf8ec-59e5-42c1-972a-cdbee6c1e9ff None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:38.332917 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bf8ec-59e5-42c1-972a-cdbee6c1e9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:38.351009 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-1f6bf8ec-59e5-42c1-972a-cdbee6c1e9ff None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:38.352172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1f6bf8ec-59e5-42c1-972a-cdbee6c1e9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:38.415686 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 578/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:38.677423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c4b01-615b-444d-b872-fa2642778183 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:38.776577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c4b01-615b-444d-b872-fa2642778183 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:38.778824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:38.810233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb43644-667c-4a6d-b0ab-f050e45e129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:38.810822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb43644-667c-4a6d-b0ab-f050e45e129d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:38.811109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb43644-667c-4a6d-b0ab-f050e45e129d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:38.811388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb43644-667c-4a6d-b0ab-f050e45e129d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:38.875444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb43644-667c-4a6d-b0ab-f050e45e129d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:38.877393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb43644-667c-4a6d-b0ab-f050e45e129d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:38.971264 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 586/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:34:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:34:38.977097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bf8ec-59e5-42c1-972a-cdbee6c1e9ff None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:38.994507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:39.070925 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6bf8ec-59e5-42c1-972a-cdbee6c1e9ff None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:39.103680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6743e25a-f129-4748-a6aa-29df8dd6b341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:39.104287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743e25a-f129-4748-a6aa-29df8dd6b341 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:39.104560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743e25a-f129-4748-a6aa-29df8dd6b341 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:39.104827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6743e25a-f129-4748-a6aa-29df8dd6b341 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:39.111215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743e25a-f129-4748-a6aa-29df8dd6b341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:39.112630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6743e25a-f129-4748-a6aa-29df8dd6b341 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-6743e25a-f129-4748-a6aa-29df8dd6b341 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:39.123450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:39.124698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:34:39.125891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 579/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 08:34:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:34:39.130907 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb43644-667c-4a6d-b0ab-f050e45e129d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:39.133895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:39.225947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb43644-667c-4a6d-b0ab-f050e45e129d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:39.258836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee4030-8f09-4ab9-9113-265a7738eda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:39.259422 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4030-8f09-4ab9-9113-265a7738eda7 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:39.259697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4030-8f09-4ab9-9113-265a7738eda7 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:39.259967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4030-8f09-4ab9-9113-265a7738eda7 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:39.260587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee4030-8f09-4ab9-9113-265a7738eda7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:39.260887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee4030-8f09-4ab9-9113-265a7738eda7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:39.262275 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee4030-8f09-4ab9-9113-265a7738eda7 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:39.269055 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 587/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Tue Oct 4 08:34:39 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:41.365169 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29659b0-9e89-47b7-a1b0-e654a2fc30f9 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:41.365713 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29659b0-9e89-47b7-a1b0-e654a2fc30f9 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:41.365954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29659b0-9e89-47b7-a1b0-e654a2fc30f9 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:41.367064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 580/1167] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:41.372144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772cb25-9864-4007-8a71-b1143cfd5065 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:41.372523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772cb25-9864-4007-8a71-b1143cfd5065 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:41.372694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772cb25-9864-4007-8a71-b1143cfd5065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:41.390715 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-c772cb25-9864-4007-8a71-b1143cfd5065 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:41.391887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c772cb25-9864-4007-8a71-b1143cfd5065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:41.451245 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:41.457073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 588/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:41.719829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29659b0-9e89-47b7-a1b0-e654a2fc30f9 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:41.795123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:41.819237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29659b0-9e89-47b7-a1b0-e654a2fc30f9 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:41.851480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1a6d7-f01a-46d7-91e4-2911f67fe2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:41.852069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1a6d7-f01a-46d7-91e4-2911f67fe2d0 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:41.852349 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1a6d7-f01a-46d7-91e4-2911f67fe2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:41.852879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1a6d7-f01a-46d7-91e4-2911f67fe2d0 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:41.853554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1a6d7-f01a-46d7-91e4-2911f67fe2d0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:41.854973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1a6d7-f01a-46d7-91e4-2911f67fe2d0 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:41.870303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 581/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Tue Oct 4 08:34:41 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:34:43.989403 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900012de-7074-431e-b8ac-a558beeb950d None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:43.989945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900012de-7074-431e-b8ac-a558beeb950d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:43.989945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900012de-7074-431e-b8ac-a558beeb950d None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:43.991193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 589/1170] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:43.996397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af26698-864e-46cf-9324-4108776a6d19 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:43.996760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af26698-864e-46cf-9324-4108776a6d19 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:43.996951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af26698-864e-46cf-9324-4108776a6d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:44.014608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-6af26698-864e-46cf-9324-4108776a6d19 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:44.015807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6af26698-864e-46cf-9324-4108776a6d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:44.079445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 582/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:44.341683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900012de-7074-431e-b8ac-a558beeb950d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:44.439412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900012de-7074-431e-b8ac-a558beeb950d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:44.472547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13132dc-2756-42f3-9d67-fa3ae16d52ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:44.473156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13132dc-2756-42f3-9d67-fa3ae16d52ba None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:44.473433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13132dc-2756-42f3-9d67-fa3ae16d52ba None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:44.473731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13132dc-2756-42f3-9d67-fa3ae16d52ba None admin] PATH_INFO: `/v3/domains/a4cc8c6fc37d4d9ab427b2b4c58b96a5` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:44.483899 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13132dc-2756-42f3-9d67-fa3ae16d52ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=a4cc8c6fc37d4d9ab427b2b4c58b96a5)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:44.485332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13132dc-2756-42f3-9d67-fa3ae16d52ba None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:44.486941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 590/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Tue Oct 4 08:34:44 2022] GET /identity/v3/domains/a4cc8c6fc37d4d9ab427b2b4c58b96a5 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:44.492604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af26698-864e-46cf-9324-4108776a6d19 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:44.555382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:44.589024 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af26698-864e-46cf-9324-4108776a6d19 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:44.621923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba83db8-a5ca-43b0-b75f-49c14754ed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:44.622466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba83db8-a5ca-43b0-b75f-49c14754ed4e None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:44.622743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba83db8-a5ca-43b0-b75f-49c14754ed4e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:44.623014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba83db8-a5ca-43b0-b75f-49c14754ed4e None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:44.623660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba83db8-a5ca-43b0-b75f-49c14754ed4e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:44.625073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba83db8-a5ca-43b0-b75f-49c14754ed4e None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:44.650642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 583/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:34:44 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:34:46.794854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d488f-20f2-46e0-81ed-152e87c9674e None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:46.795400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d488f-20f2-46e0-81ed-152e87c9674e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:46.795507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d488f-20f2-46e0-81ed-152e87c9674e None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:46.796761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 591/1174] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:46.802733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51896ac5-b734-4c02-a9af-4d89bd234f7e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:46.803160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51896ac5-b734-4c02-a9af-4d89bd234f7e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:46.803288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51896ac5-b734-4c02-a9af-4d89bd234f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:46.821637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-51896ac5-b734-4c02-a9af-4d89bd234f7e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:46.822839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-51896ac5-b734-4c02-a9af-4d89bd234f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:46.885821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 584/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:47.156041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d488f-20f2-46e0-81ed-152e87c9674e None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:47.213054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:47.255260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d488f-20f2-46e0-81ed-152e87c9674e None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:47.288485 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6929f-2629-4dbe-b886-2e9a70f7402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:47.289105 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6929f-2629-4dbe-b886-2e9a70f7402f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:47.289379 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6929f-2629-4dbe-b886-2e9a70f7402f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:47.289649 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6929f-2629-4dbe-b886-2e9a70f7402f None admin] PATH_INFO: `/v3/domains/a4cc8c6fc37d4d9ab427b2b4c58b96a5` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:47.290767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6929f-2629-4dbe-b886-2e9a70f7402f None admin] RBAC: Authorizing `identity:get_domain(domain_id=a4cc8c6fc37d4d9ab427b2b4c58b96a5)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:47.292191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6929f-2629-4dbe-b886-2e9a70f7402f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:47.293751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 592/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Tue Oct 4 08:34:47 2022] GET /identity/v3/domains/a4cc8c6fc37d4d9ab427b2b4c58b96a5 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:47.299590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51896ac5-b734-4c02-a9af-4d89bd234f7e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:47.346033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:47.395756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51896ac5-b734-4c02-a9af-4d89bd234f7e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:47.428239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df2cf4-25a9-4375-8c88-49f97a67a6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:47.428820 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df2cf4-25a9-4375-8c88-49f97a67a6c4 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:47.429108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df2cf4-25a9-4375-8c88-49f97a67a6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:47.429379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df2cf4-25a9-4375-8c88-49f97a67a6c4 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:47.430053 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df2cf4-25a9-4375-8c88-49f97a67a6c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:47.431464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df2cf4-25a9-4375-8c88-49f97a67a6c4 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:47.453095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 585/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:34:47 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:34:49.621089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f6dc2-12ce-43c4-8d45-3f415715907b None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:49.621711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f6dc2-12ce-43c4-8d45-3f415715907b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:49.621711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f6dc2-12ce-43c4-8d45-3f415715907b None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:49.623058 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 593/1178] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:49.628662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ffcff-42a1-4804-ab00-62bd14fcbd3a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:49.629079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ffcff-42a1-4804-ab00-62bd14fcbd3a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:49.629215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ffcff-42a1-4804-ab00-62bd14fcbd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:49.648310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-cb1ffcff-42a1-4804-ab00-62bd14fcbd3a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:49.649498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-cb1ffcff-42a1-4804-ab00-62bd14fcbd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:49.713300 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 586/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:49.981143 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f6dc2-12ce-43c4-8d45-3f415715907b None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:50.085920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f6dc2-12ce-43c4-8d45-3f415715907b None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:50.111182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:50.119535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b22386-15ce-4db5-8cb1-7482785b8a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:50.120147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b22386-15ce-4db5-8cb1-7482785b8a45 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:50.120425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b22386-15ce-4db5-8cb1-7482785b8a45 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:50.120694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b22386-15ce-4db5-8cb1-7482785b8a45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:50.130388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b22386-15ce-4db5-8cb1-7482785b8a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-f1b22386-15ce-4db5-8cb1-7482785b8a45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:34:50.134993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:50.136121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:34:50.136976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 594/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:34:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:34:50.142056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ffcff-42a1-4804-ab00-62bd14fcbd3a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:50.173989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:50.237125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ffcff-42a1-4804-ab00-62bd14fcbd3a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:50.271617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc295b-90b6-44e9-b020-fa48e64cba95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:50.272217 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc295b-90b6-44e9-b020-fa48e64cba95 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:50.272492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc295b-90b6-44e9-b020-fa48e64cba95 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:50.272757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc295b-90b6-44e9-b020-fa48e64cba95 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:50.273399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc295b-90b6-44e9-b020-fa48e64cba95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:50.273706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc295b-90b6-44e9-b020-fa48e64cba95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:50.275070 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc295b-90b6-44e9-b020-fa48e64cba95 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:50.283278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 587/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:34:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:50.289296 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b22386-15ce-4db5-8cb1-7482785b8a45 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:50.386067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b22386-15ce-4db5-8cb1-7482785b8a45 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:50.419055 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda8661-58a9-4b5e-ad82-a3e0890066e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:50.419657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda8661-58a9-4b5e-ad82-a3e0890066e8 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:50.419931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda8661-58a9-4b5e-ad82-a3e0890066e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:50.420199 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda8661-58a9-4b5e-ad82-a3e0890066e8 None admin] PATH_INFO: `/v3/users/e16b134fc2ef440e8f5acd61eb4213f5` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:50.432641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda8661-58a9-4b5e-ad82-a3e0890066e8 None admin] RBAC: Authorizing `identity:get_user(user_id=e16b134fc2ef440e8f5acd61eb4213f5)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:50.434104 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda8661-58a9-4b5e-ad82-a3e0890066e8 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:50.435652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 595/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:50 2022] GET /identity/v3/users/e16b134fc2ef440e8f5acd61eb4213f5 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:50.441343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc295b-90b6-44e9-b020-fa48e64cba95 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:50.533397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:50.540076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc295b-90b6-44e9-b020-fa48e64cba95 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:50.573450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721d50e-fce5-4ab7-bb83-89860aedb8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:50.574094 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721d50e-fce5-4ab7-bb83-89860aedb8c9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:50.574374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721d50e-fce5-4ab7-bb83-89860aedb8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:50.574647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721d50e-fce5-4ab7-bb83-89860aedb8c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:50.640010 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721d50e-fce5-4ab7-bb83-89860aedb8c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:50.641451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721d50e-fce5-4ab7-bb83-89860aedb8c9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:50.730971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:50.737957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 588/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:34:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:34:50.743991 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda8661-58a9-4b5e-ad82-a3e0890066e8 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:50.788288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:50.840429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda8661-58a9-4b5e-ad82-a3e0890066e8 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:50.872693 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693489f8-9703-42bf-be5b-c79e7849f4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:50.873310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693489f8-9703-42bf-be5b-c79e7849f4bb None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:50.873591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693489f8-9703-42bf-be5b-c79e7849f4bb None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:50.873892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693489f8-9703-42bf-be5b-c79e7849f4bb None admin] PATH_INFO: `/v3/projects/a73654b9958848e38a539a598ca03dca` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:50.875005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693489f8-9703-42bf-be5b-c79e7849f4bb None admin] RBAC: Authorizing `identity:get_project(project_id=a73654b9958848e38a539a598ca03dca)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:50.876976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693489f8-9703-42bf-be5b-c79e7849f4bb None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:50.878599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 596/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:34:50 2022] GET /identity/v3/projects/a73654b9958848e38a539a598ca03dca => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:50.884802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721d50e-fce5-4ab7-bb83-89860aedb8c9 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:50.979552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721d50e-fce5-4ab7-bb83-89860aedb8c9 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:51.011865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87b172-588d-4860-836a-d840b0018e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:51.012448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87b172-588d-4860-836a-d840b0018e3f None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:51.012728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87b172-588d-4860-836a-d840b0018e3f None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:51.012997 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87b172-588d-4860-836a-d840b0018e3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:51.013721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87b172-588d-4860-836a-d840b0018e3f None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=a73654b9958848e38a539a598ca03dca, user.id=e16b134fc2ef440e8f5acd61eb4213f5) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:51.014031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87b172-588d-4860-836a-d840b0018e3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:51.015436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87b172-588d-4860-836a-d840b0018e3f None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:51.020538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 589/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:34:50 2022] GET /identity/v3/role_assignments?user.id=e16b134fc2ef440e8f5acd61eb4213f5&scope.project.id=a73654b9958848e38a539a598ca03dca&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:53.104314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56aa723-6995-4e9e-9dce-ad75f12971d3 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:53.104865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56aa723-6995-4e9e-9dce-ad75f12971d3 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:53.104865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56aa723-6995-4e9e-9dce-ad75f12971d3 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:53.106243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 597/1186] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:53.111838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfa1e3-6c37-472b-9cd6-9c082bda6dc5 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:53.112212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfa1e3-6c37-472b-9cd6-9c082bda6dc5 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:53.112381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfa1e3-6c37-472b-9cd6-9c082bda6dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:53.129967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-4adfa1e3-6c37-472b-9cd6-9c082bda6dc5 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:53.131127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4adfa1e3-6c37-472b-9cd6-9c082bda6dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:53.184345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:53.195734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 590/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:53.462570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56aa723-6995-4e9e-9dce-ad75f12971d3 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:53.560371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56aa723-6995-4e9e-9dce-ad75f12971d3 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:53.573426 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:53.594082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6e03d-5a0e-482d-9622-0a72a6905ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:53.594730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6e03d-5a0e-482d-9622-0a72a6905ba5 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:53.595013 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6e03d-5a0e-482d-9622-0a72a6905ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:53.595286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6e03d-5a0e-482d-9622-0a72a6905ba5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:53.605233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6e03d-5a0e-482d-9622-0a72a6905ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-11c6e03d-5a0e-482d-9622-0a72a6905ba5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:34:53.609978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:53.611739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:34:53.612872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 598/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:34:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:34:53.617445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfa1e3-6c37-472b-9cd6-9c082bda6dc5 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:53.711457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfa1e3-6c37-472b-9cd6-9c082bda6dc5 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:53.743693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60021abc-db16-4af6-b239-8e1744262057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:53.744223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60021abc-db16-4af6-b239-8e1744262057 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:53.744498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60021abc-db16-4af6-b239-8e1744262057 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:53.744765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60021abc-db16-4af6-b239-8e1744262057 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:53.745411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60021abc-db16-4af6-b239-8e1744262057 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:53.745728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60021abc-db16-4af6-b239-8e1744262057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:53.747078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60021abc-db16-4af6-b239-8e1744262057 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:53.755304 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 591/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:34:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:53.760987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6e03d-5a0e-482d-9622-0a72a6905ba5 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:53.857111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6e03d-5a0e-482d-9622-0a72a6905ba5 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:53.889066 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78622d3-d0c6-47ac-8d97-9adc7253db5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:53.889663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78622d3-d0c6-47ac-8d97-9adc7253db5d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:53.889968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78622d3-d0c6-47ac-8d97-9adc7253db5d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:53.890237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78622d3-d0c6-47ac-8d97-9adc7253db5d None admin] PATH_INFO: `/v3/users/e16b134fc2ef440e8f5acd61eb4213f5` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:53.891325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78622d3-d0c6-47ac-8d97-9adc7253db5d None admin] RBAC: Authorizing `identity:get_user(user_id=e16b134fc2ef440e8f5acd61eb4213f5)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:53.893337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78622d3-d0c6-47ac-8d97-9adc7253db5d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:53.894885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 599/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:53 2022] GET /identity/v3/users/e16b134fc2ef440e8f5acd61eb4213f5 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:53.900533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60021abc-db16-4af6-b239-8e1744262057 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:53.950200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:53.994774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60021abc-db16-4af6-b239-8e1744262057 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:54.026755 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc5e47-db5d-4c16-a3df-cf1a8dee62df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:54.027357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc5e47-db5d-4c16-a3df-cf1a8dee62df None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:54.027631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc5e47-db5d-4c16-a3df-cf1a8dee62df None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:54.027897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc5e47-db5d-4c16-a3df-cf1a8dee62df None admin] PATH_INFO: `/v3/projects/a73654b9958848e38a539a598ca03dca` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:54.028981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc5e47-db5d-4c16-a3df-cf1a8dee62df None admin] RBAC: Authorizing `identity:get_project(project_id=a73654b9958848e38a539a598ca03dca)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:54.030409 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc5e47-db5d-4c16-a3df-cf1a8dee62df None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:54.031977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 592/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:34:53 2022] GET /identity/v3/projects/a73654b9958848e38a539a598ca03dca => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:54.037845 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78622d3-d0c6-47ac-8d97-9adc7253db5d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:54.084516 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:54.139915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78622d3-d0c6-47ac-8d97-9adc7253db5d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:54.171994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47db4f-44b6-432e-8f71-6181b5aa42e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:54.172629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47db4f-44b6-432e-8f71-6181b5aa42e8 None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:54.173439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47db4f-44b6-432e-8f71-6181b5aa42e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:54.173733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47db4f-44b6-432e-8f71-6181b5aa42e8 None admin] PATH_INFO: `/v3/projects/a73654b9958848e38a539a598ca03dca/users/e16b134fc2ef440e8f5acd61eb4213f5/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:54.175596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47db4f-44b6-432e-8f71-6181b5aa42e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a73654b9958848e38a539a598ca03dca, user_id=e16b134fc2ef440e8f5acd61eb4213f5, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:54.177018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47db4f-44b6-432e-8f71-6181b5aa42e8 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:54.189809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 600/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:34:54 2022] PUT /identity/v3/projects/a73654b9958848e38a539a598ca03dca/users/e16b134fc2ef440e8f5acd61eb4213f5/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:34:56.281120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac65caa-8150-4084-8058-e416e67de794 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:56.281680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac65caa-8150-4084-8058-e416e67de794 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:56.281680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac65caa-8150-4084-8058-e416e67de794 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:56.283543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 593/1193] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:34:56.288649 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a90856-8b40-482e-b91c-f75d581dcd0e None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:56.289018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a90856-8b40-482e-b91c-f75d581dcd0e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:56.289209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a90856-8b40-482e-b91c-f75d581dcd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:56.308304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-a9a90856-8b40-482e-b91c-f75d581dcd0e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:34:56.309475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a9a90856-8b40-482e-b91c-f75d581dcd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:34:56.373250 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 601/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:34:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:34:56.642629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac65caa-8150-4084-8058-e416e67de794 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:56.743157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac65caa-8150-4084-8058-e416e67de794 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:56.762249 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:56.775879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1d19f-affe-47c6-8c4e-94fc87889b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:56.776493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1d19f-affe-47c6-8c4e-94fc87889b12 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:56.776769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1d19f-affe-47c6-8c4e-94fc87889b12 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:56.777038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1d19f-affe-47c6-8c4e-94fc87889b12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:56.787587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1d19f-affe-47c6-8c4e-94fc87889b12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-a5b1d19f-affe-47c6-8c4e-94fc87889b12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:34:56.792258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:34:56.793366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:34:56.794267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 594/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:34:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:34:56.799890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a90856-8b40-482e-b91c-f75d581dcd0e None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:56.828034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:56.895778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a90856-8b40-482e-b91c-f75d581dcd0e None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:56.928578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc29211-9264-4b4d-ba57-c62290f5d972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:56.929167 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc29211-9264-4b4d-ba57-c62290f5d972 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:56.929442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc29211-9264-4b4d-ba57-c62290f5d972 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:56.929729 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc29211-9264-4b4d-ba57-c62290f5d972 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:56.930380 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc29211-9264-4b4d-ba57-c62290f5d972 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:56.930682 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc29211-9264-4b4d-ba57-c62290f5d972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:56.932036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc29211-9264-4b4d-ba57-c62290f5d972 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:56.939651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 602/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:34:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:56.945506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1d19f-affe-47c6-8c4e-94fc87889b12 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:57.040276 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1d19f-affe-47c6-8c4e-94fc87889b12 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:57.071856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fda665-0f9c-43f9-8b6b-b87a2b681a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:57.072457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fda665-0f9c-43f9-8b6b-b87a2b681a45 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:57.072733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fda665-0f9c-43f9-8b6b-b87a2b681a45 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:57.073007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fda665-0f9c-43f9-8b6b-b87a2b681a45 None admin] PATH_INFO: `/v3/users/e16b134fc2ef440e8f5acd61eb4213f5` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:57.074120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fda665-0f9c-43f9-8b6b-b87a2b681a45 None admin] RBAC: Authorizing `identity:get_user(user_id=e16b134fc2ef440e8f5acd61eb4213f5)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:57.075526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fda665-0f9c-43f9-8b6b-b87a2b681a45 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:57.077033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 595/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Tue Oct 4 08:34:56 2022] GET /identity/v3/users/e16b134fc2ef440e8f5acd61eb4213f5 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:57.082496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc29211-9264-4b4d-ba57-c62290f5d972 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:57.170834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:57.179590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc29211-9264-4b4d-ba57-c62290f5d972 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:57.211734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e13e89-9314-4bd3-aaef-48f1269419f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:57.212318 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e13e89-9314-4bd3-aaef-48f1269419f9 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:57.212594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e13e89-9314-4bd3-aaef-48f1269419f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:57.212862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e13e89-9314-4bd3-aaef-48f1269419f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:57.276620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e13e89-9314-4bd3-aaef-48f1269419f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:57.278077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e13e89-9314-4bd3-aaef-48f1269419f9 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:57.362991 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:57.372507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 603/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:34:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:34:57.378924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fda665-0f9c-43f9-8b6b-b87a2b681a45 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:57.425427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:34:57.473258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fda665-0f9c-43f9-8b6b-b87a2b681a45 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:57.505736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd98c5b-cb49-451a-b452-10041dcea8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:57.506345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd98c5b-cb49-451a-b452-10041dcea8b7 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:57.506623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd98c5b-cb49-451a-b452-10041dcea8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:57.506894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd98c5b-cb49-451a-b452-10041dcea8b7 None admin] PATH_INFO: `/v3/projects/a73654b9958848e38a539a598ca03dca` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:57.507975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd98c5b-cb49-451a-b452-10041dcea8b7 None admin] RBAC: Authorizing `identity:get_project(project_id=a73654b9958848e38a539a598ca03dca)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:57.509380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd98c5b-cb49-451a-b452-10041dcea8b7 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:57.510988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 596/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Tue Oct 4 08:34:57 2022] GET /identity/v3/projects/a73654b9958848e38a539a598ca03dca => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:34:57.517007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e13e89-9314-4bd3-aaef-48f1269419f9 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:34:57.613504 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e13e89-9314-4bd3-aaef-48f1269419f9 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:34:57.646122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc941f33-a350-4c1d-8de2-580daf07633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:34:57.646707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc941f33-a350-4c1d-8de2-580daf07633a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:34:57.646988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc941f33-a350-4c1d-8de2-580daf07633a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:34:57.647261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc941f33-a350-4c1d-8de2-580daf07633a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:34:57.647978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc941f33-a350-4c1d-8de2-580daf07633a None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=a73654b9958848e38a539a598ca03dca, user.id=e16b134fc2ef440e8f5acd61eb4213f5) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:34:57.648288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc941f33-a350-4c1d-8de2-580daf07633a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:34:57.649733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc941f33-a350-4c1d-8de2-580daf07633a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:34:57.655448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 604/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:34:57 2022] GET /identity/v3/role_assignments?user.id=e16b134fc2ef440e8f5acd61eb4213f5&scope.project.id=a73654b9958848e38a539a598ca03dca&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:35:07.501217 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=42 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:07.652809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=72 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:52.883129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e02e5-c1cd-474f-8ed6-10c4806a9091 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:52.883762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e02e5-c1cd-474f-8ed6-10c4806a9091 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:52.883762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e02e5-c1cd-474f-8ed6-10c4806a9091 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:52.884902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 597/1201] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:35:52.893876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350488d1-ac89-4313-a22c-b8ed6d334c3b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:52.893876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350488d1-ac89-4313-a22c-b8ed6d334c3b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:52.893876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350488d1-ac89-4313-a22c-b8ed6d334c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:52.913064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-350488d1-ac89-4313-a22c-b8ed6d334c3b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:35:52.914346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-350488d1-ac89-4313-a22c-b8ed6d334c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:35:52.995345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 605/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:35:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:35:53.340394 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e02e5-c1cd-474f-8ed6-10c4806a9091 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:53.479447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e02e5-c1cd-474f-8ed6-10c4806a9091 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:53.526210 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc89b3-a396-4fcc-9858-896519143d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:53.526887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc89b3-a396-4fcc-9858-896519143d0b None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:53.527331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc89b3-a396-4fcc-9858-896519143d0b None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:53.527922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc89b3-a396-4fcc-9858-896519143d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:53.539563 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc89b3-a396-4fcc-9858-896519143d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:53.541397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc89b3-a396-4fcc-9858-896519143d0b None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-c1bc89b3-a396-4fcc-9858-896519143d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:35:53.558822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:35:53.560163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:35:53.561133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:35:53.561133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:35:53.561133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:35:53.568093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 598/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:35:53.568270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350488d1-ac89-4313-a22c-b8ed6d334c3b None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:53.693740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350488d1-ac89-4313-a22c-b8ed6d334c3b None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:53.736632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:53.746116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947c654-e0d2-4a72-aa5c-6a39d623a838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:53.746941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947c654-e0d2-4a72-aa5c-6a39d623a838 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:53.747253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947c654-e0d2-4a72-aa5c-6a39d623a838 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:53.747677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947c654-e0d2-4a72-aa5c-6a39d623a838 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:53.748515 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947c654-e0d2-4a72-aa5c-6a39d623a838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:35:53.748904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947c654-e0d2-4a72-aa5c-6a39d623a838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:53.750686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947c654-e0d2-4a72-aa5c-6a39d623a838 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:53.772502 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 606/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:35:53.775295 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc89b3-a396-4fcc-9858-896519143d0b None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:53.844861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:53.914262 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc89b3-a396-4fcc-9858-896519143d0b None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:53.957638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68d289-40de-4478-80b1-3c60211b9131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:53.958565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68d289-40de-4478-80b1-3c60211b9131 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:53.958826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68d289-40de-4478-80b1-3c60211b9131 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:53.959103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68d289-40de-4478-80b1-3c60211b9131 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:53.959753 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68d289-40de-4478-80b1-3c60211b9131 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:53.961744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68d289-40de-4478-80b1-3c60211b9131 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:54.001805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 599/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Tue Oct 4 08:35:53 2022] POST /identity/v3/users => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:35:56.356235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b0940-17bc-44d7-ab54-c7db20051d38 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:56.357446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b0940-17bc-44d7-ab54-c7db20051d38 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:56.357446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b0940-17bc-44d7-ab54-c7db20051d38 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:56.358730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 607/1206] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:35:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:35:56.367935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ee1e5-c544-40b4-b542-74536e0c098f None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:56.368439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ee1e5-c544-40b4-b542-74536e0c098f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:56.368542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ee1e5-c544-40b4-b542-74536e0c098f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:56.388978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d89ee1e5-c544-40b4-b542-74536e0c098f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:35:56.390580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d89ee1e5-c544-40b4-b542-74536e0c098f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:35:56.468621 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 600/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:35:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:35:56.769303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b0940-17bc-44d7-ab54-c7db20051d38 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:56.914091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b0940-17bc-44d7-ab54-c7db20051d38 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:56.962378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad36a1-e8d4-4a21-b27b-7ed5d35b1476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:56.966624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad36a1-e8d4-4a21-b27b-7ed5d35b1476 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:56.966624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad36a1-e8d4-4a21-b27b-7ed5d35b1476 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:56.966624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad36a1-e8d4-4a21-b27b-7ed5d35b1476 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:56.986285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad36a1-e8d4-4a21-b27b-7ed5d35b1476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-8bad36a1-e8d4-4a21-b27b-7ed5d35b1476 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:35:56.991604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:35:56.992659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:35:56.993568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 608/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:35:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:35:56.999815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ee1e5-c544-40b4-b542-74536e0c098f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:57.049065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:57.131823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ee1e5-c544-40b4-b542-74536e0c098f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:57.175288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba4695-e92b-4059-b39b-97c6e716f83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:57.176153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba4695-e92b-4059-b39b-97c6e716f83c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:57.176430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba4695-e92b-4059-b39b-97c6e716f83c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:57.176726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba4695-e92b-4059-b39b-97c6e716f83c None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:57.177357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba4695-e92b-4059-b39b-97c6e716f83c None admin] RBAC: Adding query filter params (name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:35:57.177658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba4695-e92b-4059-b39b-97c6e716f83c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:57.179135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba4695-e92b-4059-b39b-97c6e716f83c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:57.188255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 601/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:35:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:35:57.194172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad36a1-e8d4-4a21-b27b-7ed5d35b1476 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:57.255557 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:57.332750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad36a1-e8d4-4a21-b27b-7ed5d35b1476 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:57.377354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64060d-81df-4646-b205-ba174389938a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:57.378194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64060d-81df-4646-b205-ba174389938a None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:57.378299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64060d-81df-4646-b205-ba174389938a None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:57.378509 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64060d-81df-4646-b205-ba174389938a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:57.389666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64060d-81df-4646-b205-ba174389938a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:57.391138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64060d-81df-4646-b205-ba174389938a None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-8a64060d-81df-4646-b205-ba174389938a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:35:57.407794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:35:57.408853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:35:57.409786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:35:57.409786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:35:57.409786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:35:57.409786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 609/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:35:57.415543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba4695-e92b-4059-b39b-97c6e716f83c None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:57.542034 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:57.546581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba4695-e92b-4059-b39b-97c6e716f83c None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:57.591781 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d178c-477d-4991-8e2a-7aa540732ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:57.592405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d178c-477d-4991-8e2a-7aa540732ea6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:57.592693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d178c-477d-4991-8e2a-7aa540732ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:57.592978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d178c-477d-4991-8e2a-7aa540732ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:57.593875 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d178c-477d-4991-8e2a-7aa540732ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:35:57.594496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d178c-477d-4991-8e2a-7aa540732ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:57.596969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d178c-477d-4991-8e2a-7aa540732ea6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:57.614013 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 602/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:35:57.621134 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64060d-81df-4646-b205-ba174389938a None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:57.738626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64060d-81df-4646-b205-ba174389938a None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:57.741762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:57.794894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f57be8-cea6-4dd7-8ec7-d25be6dd0704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:57.795497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f57be8-cea6-4dd7-8ec7-d25be6dd0704 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:57.795777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f57be8-cea6-4dd7-8ec7-d25be6dd0704 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:57.796052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f57be8-cea6-4dd7-8ec7-d25be6dd0704 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:57.807028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f57be8-cea6-4dd7-8ec7-d25be6dd0704 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:57.808868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f57be8-cea6-4dd7-8ec7-d25be6dd0704 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-35f57be8-cea6-4dd7-8ec7-d25be6dd0704 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:35:57.823071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:35:57.824102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:35:57.825089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:35:57.825089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:35:57.825089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:35:57.825089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:35:57.825089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:35:57.825089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:35:57.825089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:35:57.825089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:35:57.825089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:35:57.825089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:35:57.830497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d178c-477d-4991-8e2a-7aa540732ea6 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:57.833335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 610/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:35:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:35:57.951865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d178c-477d-4991-8e2a-7aa540732ea6 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:57.991972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fd64a-17a4-41a6-ad38-d884b8937e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:57.992580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fd64a-17a4-41a6-ad38-d884b8937e87 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:57.992860 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fd64a-17a4-41a6-ad38-d884b8937e87 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:57.993127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fd64a-17a4-41a6-ad38-d884b8937e87 None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:57.993893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fd64a-17a4-41a6-ad38-d884b8937e87 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:35:57.994196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fd64a-17a4-41a6-ad38-d884b8937e87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:57.995610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fd64a-17a4-41a6-ad38-d884b8937e87 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:58.014393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 603/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Oct 4 08:35:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:35:58.020191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f57be8-cea6-4dd7-8ec7-d25be6dd0704 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:58.139299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f57be8-cea6-4dd7-8ec7-d25be6dd0704 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:58.175966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cd6ea-6215-42f0-a572-4c30f80f0edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:58.176905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cd6ea-6215-42f0-a572-4c30f80f0edb None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:58.177349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cd6ea-6215-42f0-a572-4c30f80f0edb None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:58.177815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cd6ea-6215-42f0-a572-4c30f80f0edb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:58.210211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:58.249953 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cd6ea-6215-42f0-a572-4c30f80f0edb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:58.251465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cd6ea-6215-42f0-a572-4c30f80f0edb None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:58.364175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 611/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:35:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:35:58.370577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fd64a-17a4-41a6-ad38-d884b8937e87 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:58.393849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:58.487973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fd64a-17a4-41a6-ad38-d884b8937e87 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:58.526511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe7326-931c-48a4-973a-6cfe2b1d3cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:58.527144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe7326-931c-48a4-973a-6cfe2b1d3cb6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:58.527420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe7326-931c-48a4-973a-6cfe2b1d3cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:58.527685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe7326-931c-48a4-973a-6cfe2b1d3cb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:58.539627 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe7326-931c-48a4-973a-6cfe2b1d3cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:58.541251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe7326-931c-48a4-973a-6cfe2b1d3cb6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-14fe7326-931c-48a4-973a-6cfe2b1d3cb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:35:58.558430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:35:58.560230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:35:58.561314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:35:58.561314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:35:58.561314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:35:58.561314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 604/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:35:58.566371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cd6ea-6215-42f0-a572-4c30f80f0edb None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:58.635970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:58.688225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cd6ea-6215-42f0-a572-4c30f80f0edb None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:58.732248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086fd42-5cc4-4ff7-9581-42bc076798a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:58.732982 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086fd42-5cc4-4ff7-9581-42bc076798a2 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:58.733344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086fd42-5cc4-4ff7-9581-42bc076798a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:58.733647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086fd42-5cc4-4ff7-9581-42bc076798a2 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:58.734455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086fd42-5cc4-4ff7-9581-42bc076798a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:35:58.734851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086fd42-5cc4-4ff7-9581-42bc076798a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:58.736619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086fd42-5cc4-4ff7-9581-42bc076798a2 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:58.751127 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 612/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:35:58.758522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe7326-931c-48a4-973a-6cfe2b1d3cb6 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:58.851285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:58.885799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe7326-931c-48a4-973a-6cfe2b1d3cb6 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:58.926873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8f195-4a0e-4c4d-8fa6-814568198ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:58.927895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8f195-4a0e-4c4d-8fa6-814568198ffb None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:58.928357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8f195-4a0e-4c4d-8fa6-814568198ffb None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:58.928832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8f195-4a0e-4c4d-8fa6-814568198ffb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:58.936248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8f195-4a0e-4c4d-8fa6-814568198ffb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:58.937928 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8f195-4a0e-4c4d-8fa6-814568198ffb None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-6ce8f195-4a0e-4c4d-8fa6-814568198ffb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:35:58.949401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:35:58.951215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:35:58.952309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:35:58.952309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:35:58.952309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:35:58.952309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:35:58.952309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:35:58.952309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:35:58.952309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:35:58.952309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:35:58.952309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 605/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:35:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:35:58.957080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086fd42-5cc4-4ff7-9581-42bc076798a2 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:59.079553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086fd42-5cc4-4ff7-9581-42bc076798a2 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:59.113354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:59.129424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835e20d-f18d-4a4b-89d7-570c2d34b6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:59.129424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835e20d-f18d-4a4b-89d7-570c2d34b6fc None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:59.129424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835e20d-f18d-4a4b-89d7-570c2d34b6fc None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:59.129424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835e20d-f18d-4a4b-89d7-570c2d34b6fc None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:59.134092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835e20d-f18d-4a4b-89d7-570c2d34b6fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:35:59.134092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835e20d-f18d-4a4b-89d7-570c2d34b6fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:59.134092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835e20d-f18d-4a4b-89d7-570c2d34b6fc None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:59.147578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 613/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:35:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:35:59.154430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8f195-4a0e-4c4d-8fa6-814568198ffb None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:35:59.262443 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8f195-4a0e-4c4d-8fa6-814568198ffb None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:35:59.296327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:35:59.303153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50aeea-32ca-447a-aea8-c547d3a33696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:35:59.303751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50aeea-32ca-447a-aea8-c547d3a33696 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:35:59.304029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50aeea-32ca-447a-aea8-c547d3a33696 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:35:59.304297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50aeea-32ca-447a-aea8-c547d3a33696 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:35:59.305011 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50aeea-32ca-447a-aea8-c547d3a33696 None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=abd158648914409f87ce0599843f0fcf) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:35:59.305316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50aeea-32ca-447a-aea8-c547d3a33696 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:35:59.306773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50aeea-32ca-447a-aea8-c547d3a33696 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:35:59.312145 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 606/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:35:59 2022] GET /identity/v3/role_assignments?user.id=abd158648914409f87ce0599843f0fcf&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:01.581553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede4771-4764-4790-a232-e666a6d03aee None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:01.582164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede4771-4764-4790-a232-e666a6d03aee None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:01.582164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede4771-4764-4790-a232-e666a6d03aee None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:01.583735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 614/1220] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:36:01.589896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ceb09-4c57-44c7-849d-b68fd96057c6 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:01.590351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ceb09-4c57-44c7-849d-b68fd96057c6 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:01.590527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ceb09-4c57-44c7-849d-b68fd96057c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:01.611376 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-fa6ceb09-4c57-44c7-849d-b68fd96057c6 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:36:01.612780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-fa6ceb09-4c57-44c7-849d-b68fd96057c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:36:01.688377 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 607/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:36:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:36:01.998115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede4771-4764-4790-a232-e666a6d03aee None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:02.110829 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede4771-4764-4790-a232-e666a6d03aee None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:02.152535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58839-e27e-4201-a7d0-fd52c3e6ee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:02.153785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58839-e27e-4201-a7d0-fd52c3e6ee0d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:02.154095 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58839-e27e-4201-a7d0-fd52c3e6ee0d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:02.154462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58839-e27e-4201-a7d0-fd52c3e6ee0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:02.166003 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e58839-e27e-4201-a7d0-fd52c3e6ee0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-24e58839-e27e-4201-a7d0-fd52c3e6ee0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:36:02.172227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:02.173235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:02.174364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 615/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:36:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:02.179960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ceb09-4c57-44c7-849d-b68fd96057c6 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:02.287081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ceb09-4c57-44c7-849d-b68fd96057c6 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:02.312627 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:02.324581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617e8a2-4c85-4cb5-a784-056e7a747b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:02.325894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617e8a2-4c85-4cb5-a784-056e7a747b39 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:02.326252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617e8a2-4c85-4cb5-a784-056e7a747b39 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:02.326529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617e8a2-4c85-4cb5-a784-056e7a747b39 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:02.327401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617e8a2-4c85-4cb5-a784-056e7a747b39 None admin] RBAC: Adding query filter params (name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:02.327763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617e8a2-4c85-4cb5-a784-056e7a747b39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:02.329243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617e8a2-4c85-4cb5-a784-056e7a747b39 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:02.338140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 608/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:36:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:02.344477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58839-e27e-4201-a7d0-fd52c3e6ee0d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:02.396601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:02.464902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58839-e27e-4201-a7d0-fd52c3e6ee0d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:02.509299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c1a64-f3bb-4b66-9860-e426aefc0a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:02.509969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c1a64-f3bb-4b66-9860-e426aefc0a41 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:02.510271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c1a64-f3bb-4b66-9860-e426aefc0a41 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:02.510528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c1a64-f3bb-4b66-9860-e426aefc0a41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:02.521422 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c1a64-f3bb-4b66-9860-e426aefc0a41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:02.522949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c1a64-f3bb-4b66-9860-e426aefc0a41 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-388c1a64-f3bb-4b66-9860-e426aefc0a41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:36:02.537587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:02.538830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:36:02.539926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:36:02.539926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:02.539926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:02.539926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 616/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:36:02.545107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617e8a2-4c85-4cb5-a784-056e7a747b39 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:02.660430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617e8a2-4c85-4cb5-a784-056e7a747b39 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:02.698324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddb71b-51e0-404b-9b5a-4a9985e33bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:02.699075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddb71b-51e0-404b-9b5a-4a9985e33bb0 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:02.699381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddb71b-51e0-404b-9b5a-4a9985e33bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:02.699636 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddb71b-51e0-404b-9b5a-4a9985e33bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:02.700828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddb71b-51e0-404b-9b5a-4a9985e33bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:02.701104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddb71b-51e0-404b-9b5a-4a9985e33bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:02.702531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddb71b-51e0-404b-9b5a-4a9985e33bb0 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:02.714563 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 609/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:02.720850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c1a64-f3bb-4b66-9860-e426aefc0a41 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:02.824524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:02.833823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c1a64-f3bb-4b66-9860-e426aefc0a41 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:02.868620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621ef80-d731-4c80-89a6-7008ed87e920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:02.869396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621ef80-d731-4c80-89a6-7008ed87e920 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:02.869683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621ef80-d731-4c80-89a6-7008ed87e920 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:02.870070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621ef80-d731-4c80-89a6-7008ed87e920 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:02.878045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621ef80-d731-4c80-89a6-7008ed87e920 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:02.879491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621ef80-d731-4c80-89a6-7008ed87e920 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-f621ef80-d731-4c80-89a6-7008ed87e920 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:36:02.892161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:02.893538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:02.894751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:02.894751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:02.894751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:36:02.894751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:36:02.894751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:36:02.894751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:36:02.894751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:36:02.894751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:36:02.894751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:36:02.894751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:02.895360 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 617/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:02.901056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddb71b-51e0-404b-9b5a-4a9985e33bb0 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:02.953782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:03.009800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddb71b-51e0-404b-9b5a-4a9985e33bb0 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:03.047101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80141f2d-8008-4a10-bb77-6c5b11318aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:03.047722 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80141f2d-8008-4a10-bb77-6c5b11318aef None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:03.048033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80141f2d-8008-4a10-bb77-6c5b11318aef None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:03.048305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80141f2d-8008-4a10-bb77-6c5b11318aef None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:03.049013 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80141f2d-8008-4a10-bb77-6c5b11318aef None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:03.049319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80141f2d-8008-4a10-bb77-6c5b11318aef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:03.050790 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80141f2d-8008-4a10-bb77-6c5b11318aef None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:03.069606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 610/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Oct 4 08:36:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:03.075764 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621ef80-d731-4c80-89a6-7008ed87e920 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:03.186159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621ef80-d731-4c80-89a6-7008ed87e920 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:03.224937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977d135-3f2e-40a0-aa60-d1ba6526187c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:03.225959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977d135-3f2e-40a0-aa60-d1ba6526187c None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:03.226242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977d135-3f2e-40a0-aa60-d1ba6526187c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:03.226569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977d135-3f2e-40a0-aa60-d1ba6526187c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:03.239190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977d135-3f2e-40a0-aa60-d1ba6526187c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:03.240628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977d135-3f2e-40a0-aa60-d1ba6526187c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-0977d135-3f2e-40a0-aa60-d1ba6526187c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:36:03.254191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:03.255197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:36:03.256197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:36:03.256197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:03.256197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:03.256197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 618/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:03.263295 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80141f2d-8008-4a10-bb77-6c5b11318aef None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:03.365619 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80141f2d-8008-4a10-bb77-6c5b11318aef None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:03.368288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:03.406185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858df0a-2767-4282-acf9-685bc22c43dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:03.406798 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858df0a-2767-4282-acf9-685bc22c43dd None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:03.407171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858df0a-2767-4282-acf9-685bc22c43dd None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:03.407546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858df0a-2767-4282-acf9-685bc22c43dd None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:03.408288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858df0a-2767-4282-acf9-685bc22c43dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:03.408657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858df0a-2767-4282-acf9-685bc22c43dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:03.410165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858df0a-2767-4282-acf9-685bc22c43dd None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:03.422234 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 611/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:03.428607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977d135-3f2e-40a0-aa60-d1ba6526187c None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:03.438969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:03.551394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977d135-3f2e-40a0-aa60-d1ba6526187c None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:03.592209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cea42-e105-44d6-99c9-13dc79d83d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:03.592851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cea42-e105-44d6-99c9-13dc79d83d53 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:03.593131 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cea42-e105-44d6-99c9-13dc79d83d53 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:03.593406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cea42-e105-44d6-99c9-13dc79d83d53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:03.601566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cea42-e105-44d6-99c9-13dc79d83d53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:03.603142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cea42-e105-44d6-99c9-13dc79d83d53 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-ad3cea42-e105-44d6-99c9-13dc79d83d53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:36:03.615020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:03.616150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:03.617385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:36:03.617385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:36:03.617385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:36:03.617385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:36:03.617385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:36:03.617385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:36:03.617385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:36:03.617385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:03.617385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 619/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:36:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:03.623097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858df0a-2767-4282-acf9-685bc22c43dd None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:03.738032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858df0a-2767-4282-acf9-685bc22c43dd None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:03.773939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de13085-297f-4c53-94db-0a5f1d3d8caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:03.774544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13085-297f-4c53-94db-0a5f1d3d8caf None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:03.774859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13085-297f-4c53-94db-0a5f1d3d8caf None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:03.775583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de13085-297f-4c53-94db-0a5f1d3d8caf None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:03.776342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de13085-297f-4c53-94db-0a5f1d3d8caf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:03.776683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de13085-297f-4c53-94db-0a5f1d3d8caf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:03.779019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de13085-297f-4c53-94db-0a5f1d3d8caf None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:03.797743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 612/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:36:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:03.798977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cea42-e105-44d6-99c9-13dc79d83d53 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:03.869508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:03.911601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cea42-e105-44d6-99c9-13dc79d83d53 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:03.948776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db4831-ff74-435d-9572-2eaae9560125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:03.949479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db4831-ff74-435d-9572-2eaae9560125 None admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:03.949875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db4831-ff74-435d-9572-2eaae9560125 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:03.950720 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db4831-ff74-435d-9572-2eaae9560125 None admin] PATH_INFO: `/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/abd158648914409f87ce0599843f0fcf/roles/a01bc9944e4f46a4a9a2b16f510541c9` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:03.967471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db4831-ff74-435d-9572-2eaae9560125 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3df14a62714f6bab2acdfc979b655b, user_id=abd158648914409f87ce0599843f0fcf, role_id=a01bc9944e4f46a4a9a2b16f510541c9)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:03.969058 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db4831-ff74-435d-9572-2eaae9560125 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:03.982708 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 620/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:36:03 2022] PUT /identity/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/abd158648914409f87ce0599843f0fcf/roles/a01bc9944e4f46a4a9a2b16f510541c9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:36:06.312610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ae5ee-03b3-4794-b510-18907782523f None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:06.313140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ae5ee-03b3-4794-b510-18907782523f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:06.313455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ae5ee-03b3-4794-b510-18907782523f None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:06.315624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 613/1233] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:36:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:36:06.323573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966b0e2-ff8e-49ba-a825-41c1777c5901 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:06.324021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966b0e2-ff8e-49ba-a825-41c1777c5901 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:06.324450 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966b0e2-ff8e-49ba-a825-41c1777c5901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:06.348904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-1966b0e2-ff8e-49ba-a825-41c1777c5901 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:36:06.350398 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1966b0e2-ff8e-49ba-a825-41c1777c5901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:36:06.437003 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 621/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:36:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:36:06.709540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ae5ee-03b3-4794-b510-18907782523f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:06.735576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:06.815261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ae5ee-03b3-4794-b510-18907782523f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:06.852767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea9c73-f4fd-424e-bdb3-30a827700db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:06.853522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea9c73-f4fd-424e-bdb3-30a827700db5 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:06.853831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea9c73-f4fd-424e-bdb3-30a827700db5 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:06.854104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea9c73-f4fd-424e-bdb3-30a827700db5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:06.865138 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea9c73-f4fd-424e-bdb3-30a827700db5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-5bea9c73-f4fd-424e-bdb3-30a827700db5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:36:06.870028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:06.872671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:06.873660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 614/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Tue Oct 4 08:36:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:06.877642 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966b0e2-ff8e-49ba-a825-41c1777c5901 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:06.935658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:07.001581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966b0e2-ff8e-49ba-a825-41c1777c5901 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:07.041203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56eac1-f56e-47cb-8934-d844b8933b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:07.041845 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56eac1-f56e-47cb-8934-d844b8933b74 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:07.042177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56eac1-f56e-47cb-8934-d844b8933b74 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:07.042408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56eac1-f56e-47cb-8934-d844b8933b74 None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:07.043130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56eac1-f56e-47cb-8934-d844b8933b74 None admin] RBAC: Adding query filter params (name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:07.043509 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56eac1-f56e-47cb-8934-d844b8933b74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:07.045067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56eac1-f56e-47cb-8934-d844b8933b74 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:07.054314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 622/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Tue Oct 4 08:36:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:07.060356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea9c73-f4fd-424e-bdb3-30a827700db5 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:07.154933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:07.171083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea9c73-f4fd-424e-bdb3-30a827700db5 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:07.205777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba301e50-0a56-4f3a-9032-0c4a754371f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:07.206378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba301e50-0a56-4f3a-9032-0c4a754371f0 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:07.206656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba301e50-0a56-4f3a-9032-0c4a754371f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:07.206919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba301e50-0a56-4f3a-9032-0c4a754371f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:07.218005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba301e50-0a56-4f3a-9032-0c4a754371f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:07.219444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba301e50-0a56-4f3a-9032-0c4a754371f0 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-ba301e50-0a56-4f3a-9032-0c4a754371f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:36:07.234935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:07.236046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:36:07.237456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:36:07.237456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:07.237456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:07.237456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 615/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:07.242783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56eac1-f56e-47cb-8934-d844b8933b74 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:07.368821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56eac1-f56e-47cb-8934-d844b8933b74 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:07.378539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:07.407994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42747f32-5e25-48ec-b426-58bb94fab5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:07.408601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42747f32-5e25-48ec-b426-58bb94fab5e2 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:07.408885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42747f32-5e25-48ec-b426-58bb94fab5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:07.409208 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42747f32-5e25-48ec-b426-58bb94fab5e2 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:07.409978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42747f32-5e25-48ec-b426-58bb94fab5e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:07.410332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42747f32-5e25-48ec-b426-58bb94fab5e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:07.411739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42747f32-5e25-48ec-b426-58bb94fab5e2 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:07.424111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 623/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:07.430146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba301e50-0a56-4f3a-9032-0c4a754371f0 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:07.538676 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba301e50-0a56-4f3a-9032-0c4a754371f0 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:07.573386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cda3e2-cbcf-4d8e-bdef-47c7e730d4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:07.574045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cda3e2-cbcf-4d8e-bdef-47c7e730d4a4 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:07.574327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cda3e2-cbcf-4d8e-bdef-47c7e730d4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:07.574602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cda3e2-cbcf-4d8e-bdef-47c7e730d4a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:07.577484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:07.583682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cda3e2-cbcf-4d8e-bdef-47c7e730d4a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:07.585384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cda3e2-cbcf-4d8e-bdef-47c7e730d4a4 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-67cda3e2-cbcf-4d8e-bdef-47c7e730d4a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:36:07.597345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:07.598478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:07.599455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:07.599455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:07.599455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:36:07.599455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:36:07.599455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:36:07.599455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:36:07.599455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:36:07.599455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:36:07.599455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:36:07.599455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:07.599455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 616/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:07.604733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42747f32-5e25-48ec-b426-58bb94fab5e2 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:07.717295 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42747f32-5e25-48ec-b426-58bb94fab5e2 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:07.755996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544259e-903d-4c8b-b2b6-6b0b8bc9179e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:07.756697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544259e-903d-4c8b-b2b6-6b0b8bc9179e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:07.757048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544259e-903d-4c8b-b2b6-6b0b8bc9179e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:07.757343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544259e-903d-4c8b-b2b6-6b0b8bc9179e None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:07.758364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544259e-903d-4c8b-b2b6-6b0b8bc9179e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:07.758721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544259e-903d-4c8b-b2b6-6b0b8bc9179e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:07.760365 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544259e-903d-4c8b-b2b6-6b0b8bc9179e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:07.780615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 624/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Oct 4 08:36:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:07.786733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cda3e2-cbcf-4d8e-bdef-47c7e730d4a4 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:07.906032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cda3e2-cbcf-4d8e-bdef-47c7e730d4a4 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:07.944371 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834b487-e578-478e-9441-f89ce62de3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:07.944976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b487-e578-478e-9441-f89ce62de3ce None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:07.945292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b487-e578-478e-9441-f89ce62de3ce None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:07.945656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834b487-e578-478e-9441-f89ce62de3ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:08.014766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:08.022778 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834b487-e578-478e-9441-f89ce62de3ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:08.024571 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834b487-e578-478e-9441-f89ce62de3ce None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:08.129733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 617/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:36:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:36:08.135561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544259e-903d-4c8b-b2b6-6b0b8bc9179e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:08.166396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:08.256758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544259e-903d-4c8b-b2b6-6b0b8bc9179e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:08.300803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61428ec5-0491-4f8e-a77d-5ecb9b88cc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:08.301637 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61428ec5-0491-4f8e-a77d-5ecb9b88cc01 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:08.302052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61428ec5-0491-4f8e-a77d-5ecb9b88cc01 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:08.302419 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61428ec5-0491-4f8e-a77d-5ecb9b88cc01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:08.317832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61428ec5-0491-4f8e-a77d-5ecb9b88cc01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:08.319906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61428ec5-0491-4f8e-a77d-5ecb9b88cc01 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-61428ec5-0491-4f8e-a77d-5ecb9b88cc01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:36:08.339001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:08.340069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:36:08.341563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:36:08.341563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:08.341563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:08.341563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 625/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:08.347203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834b487-e578-478e-9441-f89ce62de3ce None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:08.453469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:08.462150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834b487-e578-478e-9441-f89ce62de3ce None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:08.506142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a5836-0cb4-4f9a-8a9a-6920425b9cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:08.506810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a5836-0cb4-4f9a-8a9a-6920425b9cc6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:08.507142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a5836-0cb4-4f9a-8a9a-6920425b9cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:08.507414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a5836-0cb4-4f9a-8a9a-6920425b9cc6 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:08.508048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a5836-0cb4-4f9a-8a9a-6920425b9cc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:08.508350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a5836-0cb4-4f9a-8a9a-6920425b9cc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:08.510149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a5836-0cb4-4f9a-8a9a-6920425b9cc6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:08.523850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 618/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:08.531085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61428ec5-0491-4f8e-a77d-5ecb9b88cc01 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:08.628788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:08.658742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61428ec5-0491-4f8e-a77d-5ecb9b88cc01 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:08.700218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c898fc9-4892-4df1-859e-b0e114b86de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:08.700884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c898fc9-4892-4df1-859e-b0e114b86de2 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:08.701166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c898fc9-4892-4df1-859e-b0e114b86de2 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:08.701614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c898fc9-4892-4df1-859e-b0e114b86de2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:08.710058 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c898fc9-4892-4df1-859e-b0e114b86de2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:08.711588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c898fc9-4892-4df1-859e-b0e114b86de2 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-2c898fc9-4892-4df1-859e-b0e114b86de2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:36:08.723853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:08.725027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:08.726395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:36:08.726395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:36:08.726395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:36:08.726395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:36:08.726395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:36:08.726395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:36:08.726395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:36:08.726395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:08.726395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 626/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:36:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:08.732022 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a5836-0cb4-4f9a-8a9a-6920425b9cc6 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:08.844113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a5836-0cb4-4f9a-8a9a-6920425b9cc6 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:08.882343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d142798-9663-4d46-be26-c7bd5d2fccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:08.883155 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d142798-9663-4d46-be26-c7bd5d2fccd7 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:08.883368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d142798-9663-4d46-be26-c7bd5d2fccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:08.883676 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d142798-9663-4d46-be26-c7bd5d2fccd7 None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:08.884549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d142798-9663-4d46-be26-c7bd5d2fccd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:08.884858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d142798-9663-4d46-be26-c7bd5d2fccd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:08.886602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d142798-9663-4d46-be26-c7bd5d2fccd7 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:08.902664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 619/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:36:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:08.909843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c898fc9-4892-4df1-859e-b0e114b86de2 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:09.025134 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c898fc9-4892-4df1-859e-b0e114b86de2 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:09.062046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe842c10-3736-485e-b467-378be88ba890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:09.062660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe842c10-3736-485e-b467-378be88ba890 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:09.062945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe842c10-3736-485e-b467-378be88ba890 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:09.063217 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe842c10-3736-485e-b467-378be88ba890 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:09.064137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe842c10-3736-485e-b467-378be88ba890 None admin] RBAC: Adding query filter params (role.id=a01bc9944e4f46a4a9a2b16f510541c9, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=abd158648914409f87ce0599843f0fcf) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:09.064407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe842c10-3736-485e-b467-378be88ba890 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:09.066158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe842c10-3736-485e-b467-378be88ba890 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:09.077744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 627/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:36:08 2022] GET /identity/v3/role_assignments?user.id=abd158648914409f87ce0599843f0fcf&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=a01bc9944e4f46a4a9a2b16f510541c9 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:11.367230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d3d6f-61a7-4972-b324-b72f3fedaad8 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:11.367836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d3d6f-61a7-4972-b324-b72f3fedaad8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:11.367836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d3d6f-61a7-4972-b324-b72f3fedaad8 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:11.369114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 620/1247] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:36:11.374966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90552ef7-02a4-4c69-b00f-528c1bc044cc None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:11.375427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90552ef7-02a4-4c69-b00f-528c1bc044cc None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:11.375534 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90552ef7-02a4-4c69-b00f-528c1bc044cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:11.382279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:11.397023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-90552ef7-02a4-4c69-b00f-528c1bc044cc None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:36:11.398346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-90552ef7-02a4-4c69-b00f-528c1bc044cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:36:11.470759 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 628/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:36:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:36:11.757246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d3d6f-61a7-4972-b324-b72f3fedaad8 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:11.778431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:11.870725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d3d6f-61a7-4972-b324-b72f3fedaad8 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:11.909999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9f80c-332d-4a74-8b3a-0b39bec3b04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:11.910583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9f80c-332d-4a74-8b3a-0b39bec3b04e None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:11.910867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9f80c-332d-4a74-8b3a-0b39bec3b04e None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:11.911139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9f80c-332d-4a74-8b3a-0b39bec3b04e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:11.923413 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9f80c-332d-4a74-8b3a-0b39bec3b04e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-33c9f80c-332d-4a74-8b3a-0b39bec3b04e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:36:11.928436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:11.929554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:11.931072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 621/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:36:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 08:36:11.936068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90552ef7-02a4-4c69-b00f-528c1bc044cc None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:12.051848 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90552ef7-02a4-4c69-b00f-528c1bc044cc None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:12.082934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:12.088533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d764d-7759-4198-baea-573e00e2678f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:12.089154 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d764d-7759-4198-baea-573e00e2678f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:12.089457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d764d-7759-4198-baea-573e00e2678f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:12.089758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d764d-7759-4198-baea-573e00e2678f None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:12.090979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d764d-7759-4198-baea-573e00e2678f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:12.091289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d764d-7759-4198-baea-573e00e2678f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:12.092671 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d764d-7759-4198-baea-573e00e2678f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:12.101643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 629/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:36:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:12.108291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9f80c-332d-4a74-8b3a-0b39bec3b04e None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:12.190803 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:12.215758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9f80c-332d-4a74-8b3a-0b39bec3b04e None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:12.251381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc2f58-160a-4527-b160-7a4f072ea228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:12.252034 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc2f58-160a-4527-b160-7a4f072ea228 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:12.252313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc2f58-160a-4527-b160-7a4f072ea228 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:12.252585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc2f58-160a-4527-b160-7a4f072ea228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:12.266133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc2f58-160a-4527-b160-7a4f072ea228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:12.267644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc2f58-160a-4527-b160-7a4f072ea228 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-d8dc2f58-160a-4527-b160-7a4f072ea228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:36:12.289874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:12.291140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:36:12.292077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:36:12.292077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:12.292077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:12.292077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 622/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:12.297675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d764d-7759-4198-baea-573e00e2678f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:12.409440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d764d-7759-4198-baea-573e00e2678f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:12.443490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00967a13-9e8f-4873-83c9-bca9b41dcfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:12.444018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00967a13-9e8f-4873-83c9-bca9b41dcfaf None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:12.444299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00967a13-9e8f-4873-83c9-bca9b41dcfaf None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:12.444573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00967a13-9e8f-4873-83c9-bca9b41dcfaf None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:12.445218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00967a13-9e8f-4873-83c9-bca9b41dcfaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:12.445526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00967a13-9e8f-4873-83c9-bca9b41dcfaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:12.446930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00967a13-9e8f-4873-83c9-bca9b41dcfaf None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:12.459139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 630/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:12.465930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc2f58-160a-4527-b160-7a4f072ea228 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:12.566302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc2f58-160a-4527-b160-7a4f072ea228 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:12.592245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:12.601082 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af99faf-5e1c-4223-bb1f-aaccfc04ea1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:12.601804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af99faf-5e1c-4223-bb1f-aaccfc04ea1d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:12.602021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af99faf-5e1c-4223-bb1f-aaccfc04ea1d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:12.602292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af99faf-5e1c-4223-bb1f-aaccfc04ea1d None admin] PATH_INFO: `/v3/users/placement` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:12.610067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af99faf-5e1c-4223-bb1f-aaccfc04ea1d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:12.611512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af99faf-5e1c-4223-bb1f-aaccfc04ea1d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-5af99faf-5e1c-4223-bb1f-aaccfc04ea1d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:36:12.623115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:12.624285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:12.625417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:12.625417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:12.625417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:36:12.625417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:36:12.625417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:36:12.625417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:36:12.625417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:36:12.625417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:36:12.625417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:36:12.625417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:12.625417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 623/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:12.630064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00967a13-9e8f-4873-83c9-bca9b41dcfaf None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:12.689387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:12.733547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00967a13-9e8f-4873-83c9-bca9b41dcfaf None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:12.767649 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c96480-f6b4-4a22-bf14-4cc6b149fd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:12.768243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c96480-f6b4-4a22-bf14-4cc6b149fd1b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:12.768523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c96480-f6b4-4a22-bf14-4cc6b149fd1b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:12.768795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c96480-f6b4-4a22-bf14-4cc6b149fd1b None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:12.769507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c96480-f6b4-4a22-bf14-4cc6b149fd1b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:12.769843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c96480-f6b4-4a22-bf14-4cc6b149fd1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:12.771282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c96480-f6b4-4a22-bf14-4cc6b149fd1b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:12.787574 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 631/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Oct 4 08:36:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:12.793382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af99faf-5e1c-4223-bb1f-aaccfc04ea1d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:12.889582 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af99faf-5e1c-4223-bb1f-aaccfc04ea1d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:12.921556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a37c85-2d68-4ee7-96f2-f7909c0bfb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:12.922172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37c85-2d68-4ee7-96f2-f7909c0bfb67 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:12.922452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37c85-2d68-4ee7-96f2-f7909c0bfb67 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:12.922723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37c85-2d68-4ee7-96f2-f7909c0bfb67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:12.974465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:12.987543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a37c85-2d68-4ee7-96f2-f7909c0bfb67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:12.988919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a37c85-2d68-4ee7-96f2-f7909c0bfb67 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:13.083033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 624/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:36:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:36:13.088805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c96480-f6b4-4a22-bf14-4cc6b149fd1b None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:13.184976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c96480-f6b4-4a22-bf14-4cc6b149fd1b None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:13.197385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:13.218914 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dfc98-c711-4bf2-b055-6d964376901e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:13.219524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dfc98-c711-4bf2-b055-6d964376901e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:13.219803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dfc98-c711-4bf2-b055-6d964376901e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:13.220076 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5dfc98-c711-4bf2-b055-6d964376901e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:13.229951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dfc98-c711-4bf2-b055-6d964376901e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:13.231413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5dfc98-c711-4bf2-b055-6d964376901e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-dd5dfc98-c711-4bf2-b055-6d964376901e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:36:13.245387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:13.246496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:36:13.247600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:36:13.247600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:13.247600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:13.247600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 632/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:13.252170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a37c85-2d68-4ee7-96f2-f7909c0bfb67 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:13.347654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:13.359595 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a37c85-2d68-4ee7-96f2-f7909c0bfb67 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:13.393698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0dd45-f0d3-4f6d-aa8c-85b005afa1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:13.394314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0dd45-f0d3-4f6d-aa8c-85b005afa1e7 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:13.394595 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0dd45-f0d3-4f6d-aa8c-85b005afa1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:13.394868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0dd45-f0d3-4f6d-aa8c-85b005afa1e7 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:13.395503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0dd45-f0d3-4f6d-aa8c-85b005afa1e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:13.395807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0dd45-f0d3-4f6d-aa8c-85b005afa1e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:13.397164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0dd45-f0d3-4f6d-aa8c-85b005afa1e7 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:13.408075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 625/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:13.413856 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dfc98-c711-4bf2-b055-6d964376901e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:13.512467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5dfc98-c711-4bf2-b055-6d964376901e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:13.544446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e8619-011b-484b-9de0-6a2f1936ea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:13.545056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e8619-011b-484b-9de0-6a2f1936ea59 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:13.545336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e8619-011b-484b-9de0-6a2f1936ea59 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:13.545611 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e8619-011b-484b-9de0-6a2f1936ea59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:13.552005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e8619-011b-484b-9de0-6a2f1936ea59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:13.553447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e8619-011b-484b-9de0-6a2f1936ea59 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-491e8619-011b-484b-9de0-6a2f1936ea59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:36:13.564686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:13.565765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:13.566868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:36:13.566868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:36:13.566868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:36:13.566868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:36:13.566868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:36:13.566868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:36:13.566868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:36:13.566868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:13.566868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 633/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:36:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:13.571510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0dd45-f0d3-4f6d-aa8c-85b005afa1e7 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:13.666637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0dd45-f0d3-4f6d-aa8c-85b005afa1e7 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:13.699016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a9ad7-679a-4b21-affd-af6eb7913144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:13.699608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a9ad7-679a-4b21-affd-af6eb7913144 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:13.699876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a9ad7-679a-4b21-affd-af6eb7913144 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:13.700140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a9ad7-679a-4b21-affd-af6eb7913144 None admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:13.700808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a9ad7-679a-4b21-affd-af6eb7913144 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:13.701105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a9ad7-679a-4b21-affd-af6eb7913144 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:13.702532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a9ad7-679a-4b21-affd-af6eb7913144 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:13.713934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 626/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:36:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:13.719950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e8619-011b-484b-9de0-6a2f1936ea59 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:13.745325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:13.817489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e8619-011b-484b-9de0-6a2f1936ea59 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:13.850077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a6645-b873-46da-973d-296df0707648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:13.850927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a6645-b873-46da-973d-296df0707648 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:13.851208 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a6645-b873-46da-973d-296df0707648 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:13.851479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a6645-b873-46da-973d-296df0707648 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:13.852192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a6645-b873-46da-973d-296df0707648 None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=abd158648914409f87ce0599843f0fcf) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:13.852505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a6645-b873-46da-973d-296df0707648 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:13.853935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a6645-b873-46da-973d-296df0707648 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:13.859060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 634/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:36:13 2022] GET /identity/v3/role_assignments?user.id=abd158648914409f87ce0599843f0fcf&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:15.936151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddb7a8-64c9-460e-ae82-3691cfb82c31 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:15.936664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddb7a8-64c9-460e-ae82-3691cfb82c31 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:15.936664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddb7a8-64c9-460e-ae82-3691cfb82c31 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:15.937953 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 627/1261] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:36:15.942918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad90737-d64f-4678-ac35-104161c571e6 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:15.943324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad90737-d64f-4678-ac35-104161c571e6 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:15.943478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad90737-d64f-4678-ac35-104161c571e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:15.961148 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6ad90737-d64f-4678-ac35-104161c571e6 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:36:15.962374 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6ad90737-d64f-4678-ac35-104161c571e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:36:16.027247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 635/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:36:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:36:16.294306 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddb7a8-64c9-460e-ae82-3691cfb82c31 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:16.305198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:16.396387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddb7a8-64c9-460e-ae82-3691cfb82c31 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:16.429101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0979d6e-5594-403d-a67f-281aba55d978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:16.429737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0979d6e-5594-403d-a67f-281aba55d978 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:16.430020 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0979d6e-5594-403d-a67f-281aba55d978 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:16.430290 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0979d6e-5594-403d-a67f-281aba55d978 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:16.439959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0979d6e-5594-403d-a67f-281aba55d978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-e0979d6e-5594-403d-a67f-281aba55d978 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:36:16.444632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:16.445739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:16.446815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 628/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:36:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:16.451762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad90737-d64f-4678-ac35-104161c571e6 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:16.469629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:16.546811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad90737-d64f-4678-ac35-104161c571e6 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:16.578464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27b282-8321-4759-ae90-029c70c10c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:16.579050 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27b282-8321-4759-ae90-029c70c10c9f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:16.579326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27b282-8321-4759-ae90-029c70c10c9f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:16.579593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27b282-8321-4759-ae90-029c70c10c9f None admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:16.580235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27b282-8321-4759-ae90-029c70c10c9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:16.580532 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27b282-8321-4759-ae90-029c70c10c9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:16.581913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27b282-8321-4759-ae90-029c70c10c9f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:16.589582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 636/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:36:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:16.595661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0979d6e-5594-403d-a67f-281aba55d978 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:16.662309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:16.695048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0979d6e-5594-403d-a67f-281aba55d978 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:16.728077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980e936-ae9a-4074-88b0-b0b0a8e4692a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:16.728695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980e936-ae9a-4074-88b0-b0b0a8e4692a None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:16.728972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980e936-ae9a-4074-88b0-b0b0a8e4692a None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:16.729242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980e936-ae9a-4074-88b0-b0b0a8e4692a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:16.738831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980e936-ae9a-4074-88b0-b0b0a8e4692a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:16.740268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980e936-ae9a-4074-88b0-b0b0a8e4692a None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-7980e936-ae9a-4074-88b0-b0b0a8e4692a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:36:16.754250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:16.755312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:36:16.756192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:36:16.756192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:16.756192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:16.756192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 629/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:16.761129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27b282-8321-4759-ae90-029c70c10c9f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:16.839420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:16.858609 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27b282-8321-4759-ae90-029c70c10c9f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:16.890606 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a086e1-2838-4a43-91c6-e53bd533a863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:16.891199 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a086e1-2838-4a43-91c6-e53bd533a863 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:16.891479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a086e1-2838-4a43-91c6-e53bd533a863 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:16.891753 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a086e1-2838-4a43-91c6-e53bd533a863 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:16.892382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a086e1-2838-4a43-91c6-e53bd533a863 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:16.892686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a086e1-2838-4a43-91c6-e53bd533a863 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:16.894067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a086e1-2838-4a43-91c6-e53bd533a863 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:16.905256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 637/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:16.910799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980e936-ae9a-4074-88b0-b0b0a8e4692a None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:17.008002 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980e936-ae9a-4074-88b0-b0b0a8e4692a None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:17.026626 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:17.041008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cb0ef-3ada-45e3-b417-a402eafd05c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:17.041628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cb0ef-3ada-45e3-b417-a402eafd05c6 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:17.041937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cb0ef-3ada-45e3-b417-a402eafd05c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:17.042210 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cb0ef-3ada-45e3-b417-a402eafd05c6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:17.050668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cb0ef-3ada-45e3-b417-a402eafd05c6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:17.052113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603cb0ef-3ada-45e3-b417-a402eafd05c6 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-603cb0ef-3ada-45e3-b417-a402eafd05c6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:36:17.062834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:17.063969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:17.065327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:17.065327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:17.065327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:36:17.065327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:36:17.065327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:36:17.065327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:36:17.065327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:36:17.065327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:36:17.065327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:36:17.065327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:17.065327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 630/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:17.069663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a086e1-2838-4a43-91c6-e53bd533a863 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:17.166146 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a086e1-2838-4a43-91c6-e53bd533a863 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:17.198563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fcae5-0cd3-4b39-8414-af9e1ec4497d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:17.199176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fcae5-0cd3-4b39-8414-af9e1ec4497d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:17.199458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fcae5-0cd3-4b39-8414-af9e1ec4497d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:17.199729 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fcae5-0cd3-4b39-8414-af9e1ec4497d None admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:17.200469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fcae5-0cd3-4b39-8414-af9e1ec4497d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:17.200773 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fcae5-0cd3-4b39-8414-af9e1ec4497d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:17.202225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fcae5-0cd3-4b39-8414-af9e1ec4497d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:17.210454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:17.219357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 638/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Oct 4 08:36:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:17.224855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cb0ef-3ada-45e3-b417-a402eafd05c6 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:17.328512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cb0ef-3ada-45e3-b417-a402eafd05c6 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:17.361334 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1056520-f36b-413e-b108-cb7fa2844126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:17.361973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1056520-f36b-413e-b108-cb7fa2844126 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:17.362249 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1056520-f36b-413e-b108-cb7fa2844126 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:17.362515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1056520-f36b-413e-b108-cb7fa2844126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:17.372617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1056520-f36b-413e-b108-cb7fa2844126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:17.374072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1056520-f36b-413e-b108-cb7fa2844126 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-e1056520-f36b-413e-b108-cb7fa2844126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:36:17.388066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:17.389187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:36:17.390101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:36:17.390101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:17.390101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:17.390101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 631/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:17.394802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fcae5-0cd3-4b39-8414-af9e1ec4497d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:17.491268 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fcae5-0cd3-4b39-8414-af9e1ec4497d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:17.523742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fe1cc-9f21-44e2-9b1f-dc6cbc55e6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:17.524337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fe1cc-9f21-44e2-9b1f-dc6cbc55e6d7 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:17.524620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fe1cc-9f21-44e2-9b1f-dc6cbc55e6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:17.524893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fe1cc-9f21-44e2-9b1f-dc6cbc55e6d7 None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:17.525527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fe1cc-9f21-44e2-9b1f-dc6cbc55e6d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:17.525858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fe1cc-9f21-44e2-9b1f-dc6cbc55e6d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:17.527237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fe1cc-9f21-44e2-9b1f-dc6cbc55e6d7 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:17.537939 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 639/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:17.544021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1056520-f36b-413e-b108-cb7fa2844126 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:17.574533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:17.639786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1056520-f36b-413e-b108-cb7fa2844126 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:17.677375 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82ffa-2d38-4acc-85bf-b1ad068add90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:17.678148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82ffa-2d38-4acc-85bf-b1ad068add90 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:17.678501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82ffa-2d38-4acc-85bf-b1ad068add90 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:17.678841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82ffa-2d38-4acc-85bf-b1ad068add90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:17.686285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de82ffa-2d38-4acc-85bf-b1ad068add90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:17.687726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de82ffa-2d38-4acc-85bf-b1ad068add90 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-4de82ffa-2d38-4acc-85bf-b1ad068add90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:36:17.698754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:17.699784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:17.702716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:36:17.702716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:36:17.702716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:36:17.702716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:36:17.702716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:36:17.702716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:36:17.702716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:36:17.702716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:17.702716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 632/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:36:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:17.705678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fe1cc-9f21-44e2-9b1f-dc6cbc55e6d7 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:17.778039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:17.820433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fe1cc-9f21-44e2-9b1f-dc6cbc55e6d7 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:17.854638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd897ca-0229-47a8-89ae-cf2f3e645b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:17.855242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd897ca-0229-47a8-89ae-cf2f3e645b9b None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:17.856091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd897ca-0229-47a8-89ae-cf2f3e645b9b None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:17.856371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd897ca-0229-47a8-89ae-cf2f3e645b9b None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:17.857045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd897ca-0229-47a8-89ae-cf2f3e645b9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:17.857348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd897ca-0229-47a8-89ae-cf2f3e645b9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:17.858819 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd897ca-0229-47a8-89ae-cf2f3e645b9b None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:17.870555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 640/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:36:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:36:17.876516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82ffa-2d38-4acc-85bf-b1ad068add90 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:17.963986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:17.973389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82ffa-2d38-4acc-85bf-b1ad068add90 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:18.006015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e68ff-3112-4d62-9f57-a311867b35fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:18.006654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e68ff-3112-4d62-9f57-a311867b35fe None admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:18.006934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e68ff-3112-4d62-9f57-a311867b35fe None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:18.007207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e68ff-3112-4d62-9f57-a311867b35fe None admin] PATH_INFO: `/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/abd158648914409f87ce0599843f0fcf/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:18.009062 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e68ff-3112-4d62-9f57-a311867b35fe None admin] RBAC: Authorizing `identity:create_grant(project_id=fa3df14a62714f6bab2acdfc979b655b, user_id=abd158648914409f87ce0599843f0fcf, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:18.010509 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e68ff-3112-4d62-9f57-a311867b35fe None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:18.022404 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 633/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Tue Oct 4 08:36:17 2022] PUT /identity/v3/projects/fa3df14a62714f6bab2acdfc979b655b/users/abd158648914409f87ce0599843f0fcf/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:36:20.123219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d961c-37df-4e51-ac31-4912ab2879e4 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:20.123772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d961c-37df-4e51-ac31-4912ab2879e4 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:20.123772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d961c-37df-4e51-ac31-4912ab2879e4 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:20.125106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 641/1274] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:36:20.130313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811014f-9a01-4db0-bda9-6f0844eac843 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:20.130714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811014f-9a01-4db0-bda9-6f0844eac843 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:20.130865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811014f-9a01-4db0-bda9-6f0844eac843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:20.149396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-0811014f-9a01-4db0-bda9-6f0844eac843 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:36:20.150605 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0811014f-9a01-4db0-bda9-6f0844eac843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:36:20.213394 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 634/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:36:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:36:20.482051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d961c-37df-4e51-ac31-4912ab2879e4 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:20.579645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d961c-37df-4e51-ac31-4912ab2879e4 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:20.595944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:20.612898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99a259-1433-4f7d-8d8a-b12a1ea66b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:20.613513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99a259-1433-4f7d-8d8a-b12a1ea66b85 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:20.613819 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99a259-1433-4f7d-8d8a-b12a1ea66b85 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:20.614094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99a259-1433-4f7d-8d8a-b12a1ea66b85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:20.624626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99a259-1433-4f7d-8d8a-b12a1ea66b85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-2d99a259-1433-4f7d-8d8a-b12a1ea66b85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 08:36:20.629383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:20.630479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:20.631632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 642/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Tue Oct 4 08:36:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:20.636344 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811014f-9a01-4db0-bda9-6f0844eac843 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:20.714797 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:20.733109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811014f-9a01-4db0-bda9-6f0844eac843 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:20.765601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f9b67-5010-44a3-9c66-0b1fbb50f2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:20.766220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f9b67-5010-44a3-9c66-0b1fbb50f2d9 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:20.766496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f9b67-5010-44a3-9c66-0b1fbb50f2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:20.766764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f9b67-5010-44a3-9c66-0b1fbb50f2d9 None admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:20.767407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f9b67-5010-44a3-9c66-0b1fbb50f2d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:20.767706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f9b67-5010-44a3-9c66-0b1fbb50f2d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:20.769052 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f9b67-5010-44a3-9c66-0b1fbb50f2d9 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:20.776708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 635/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Tue Oct 4 08:36:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:20.782976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99a259-1433-4f7d-8d8a-b12a1ea66b85 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:20.878205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99a259-1433-4f7d-8d8a-b12a1ea66b85 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:20.911120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1878e-7194-47ca-af5b-9269cfbe092f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:20.911719 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1878e-7194-47ca-af5b-9269cfbe092f None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:20.911995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1878e-7194-47ca-af5b-9269cfbe092f None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:20.912261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1878e-7194-47ca-af5b-9269cfbe092f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:20.921852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1878e-7194-47ca-af5b-9269cfbe092f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:20.923272 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1878e-7194-47ca-af5b-9269cfbe092f None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-18a1878e-7194-47ca-af5b-9269cfbe092f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:36:20.937556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:20.938728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:36:20.939840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:36:20.939840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:20.939840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:20.939840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 643/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:20.944375 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f9b67-5010-44a3-9c66-0b1fbb50f2d9 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:21.040321 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f9b67-5010-44a3-9c66-0b1fbb50f2d9 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:21.072125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab47c93-e3a6-4f95-a8d3-e1d15b800392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:21.072718 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab47c93-e3a6-4f95-a8d3-e1d15b800392 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:21.073001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab47c93-e3a6-4f95-a8d3-e1d15b800392 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:21.073274 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab47c93-e3a6-4f95-a8d3-e1d15b800392 None admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:21.074529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab47c93-e3a6-4f95-a8d3-e1d15b800392 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:21.074846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab47c93-e3a6-4f95-a8d3-e1d15b800392 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:21.076195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab47c93-e3a6-4f95-a8d3-e1d15b800392 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:21.087330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 636/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:21.092974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1878e-7194-47ca-af5b-9269cfbe092f None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:21.123338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:21.190625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1878e-7194-47ca-af5b-9269cfbe092f None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:21.222068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda288f-e5a5-4a59-bdce-53dc2578634e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:21.222674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda288f-e5a5-4a59-bdce-53dc2578634e None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:21.222955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda288f-e5a5-4a59-bdce-53dc2578634e None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:21.223227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda288f-e5a5-4a59-bdce-53dc2578634e None admin] PATH_INFO: `/v3/users/placement` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:21.231070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda288f-e5a5-4a59-bdce-53dc2578634e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:21.232499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda288f-e5a5-4a59-bdce-53dc2578634e None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-6dda288f-e5a5-4a59-bdce-53dc2578634e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 08:36:21.243117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:21.244253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:21.245471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:21.245471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:21.245471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 08:36:21.245471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 08:36:21.245471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 08:36:21.245471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 08:36:21.245471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 08:36:21.245471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 08:36:21.245471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 08:36:21.245471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:21.245471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 644/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:21.250359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab47c93-e3a6-4f95-a8d3-e1d15b800392 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:21.254099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:21.350323 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab47c93-e3a6-4f95-a8d3-e1d15b800392 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:21.383436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440c10d-be87-4edc-b755-a507a9cbc3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:21.384021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440c10d-be87-4edc-b755-a507a9cbc3ff None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:21.384297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440c10d-be87-4edc-b755-a507a9cbc3ff None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:21.384564 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440c10d-be87-4edc-b755-a507a9cbc3ff None admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:21.385271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440c10d-be87-4edc-b755-a507a9cbc3ff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:21.385569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440c10d-be87-4edc-b755-a507a9cbc3ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:21.387039 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440c10d-be87-4edc-b755-a507a9cbc3ff None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:21.403127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 637/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Tue Oct 4 08:36:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:21.408779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda288f-e5a5-4a59-bdce-53dc2578634e None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:21.495904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:21.506470 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda288f-e5a5-4a59-bdce-53dc2578634e None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:21.538731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a0cd9-1381-4dba-9768-7892302e4b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:21.539325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a0cd9-1381-4dba-9768-7892302e4b50 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:21.539601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a0cd9-1381-4dba-9768-7892302e4b50 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:21.539871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a0cd9-1381-4dba-9768-7892302e4b50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:21.603597 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a0cd9-1381-4dba-9768-7892302e4b50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:21.604972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a0cd9-1381-4dba-9768-7892302e4b50 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:21.689584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:21.699783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 645/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Tue Oct 4 08:36:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 08:36:21.705497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440c10d-be87-4edc-b755-a507a9cbc3ff None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:21.764851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:21.802608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440c10d-be87-4edc-b755-a507a9cbc3ff None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:21.835698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9675077-7271-4e0e-a380-55323b3a7e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:21.836314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9675077-7271-4e0e-a380-55323b3a7e7c None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:21.836593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9675077-7271-4e0e-a380-55323b3a7e7c None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:21.836865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9675077-7271-4e0e-a380-55323b3a7e7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:21.846540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9675077-7271-4e0e-a380-55323b3a7e7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:21.847972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9675077-7271-4e0e-a380-55323b3a7e7c None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-a9675077-7271-4e0e-a380-55323b3a7e7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 08:36:21.862246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:21.863317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 08:36:21.864223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 08:36:21.864223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 08:36:21.864223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:21.864223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 638/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:21.868918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a0cd9-1381-4dba-9768-7892302e4b50 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:21.963078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a0cd9-1381-4dba-9768-7892302e4b50 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:21.995354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cdfff-6ca6-45d3-a27e-a18dee2aa1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:21.996560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cdfff-6ca6-45d3-a27e-a18dee2aa1af None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:21.996834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cdfff-6ca6-45d3-a27e-a18dee2aa1af None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:21.997100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cdfff-6ca6-45d3-a27e-a18dee2aa1af None admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:21.997749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cdfff-6ca6-45d3-a27e-a18dee2aa1af None admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:21.998052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cdfff-6ca6-45d3-a27e-a18dee2aa1af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:21.999396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cdfff-6ca6-45d3-a27e-a18dee2aa1af None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:22.009940 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 646/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Tue Oct 4 08:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:22.015401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9675077-7271-4e0e-a380-55323b3a7e7c None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:22.110425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9675077-7271-4e0e-a380-55323b3a7e7c None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:22.123720 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:22.142847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ac278-6841-4b26-8c31-bd4a8204a1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:22.143463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ac278-6841-4b26-8c31-bd4a8204a1f2 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:22.143742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ac278-6841-4b26-8c31-bd4a8204a1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:22.144014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ac278-6841-4b26-8c31-bd4a8204a1f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:22.154926 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ac278-6841-4b26-8c31-bd4a8204a1f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:22.156779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ac278-6841-4b26-8c31-bd4a8204a1f2 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application [None req-1e5ac278-6841-4b26-8c31-bd4a8204a1f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 08:36:22.169436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:22.170484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:22.171824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 08:36:22.171824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 08:36:22.171824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 08:36:22.171824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 08:36:22.171824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 08:36:22.171824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 08:36:22.171824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 08:36:22.171824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: ERROR keystone.server.flask.application Oct 04 08:36:22.171824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 639/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Tue Oct 4 08:36:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:22.176367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cdfff-6ca6-45d3-a27e-a18dee2aa1af None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:22.224573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:22.273876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cdfff-6ca6-45d3-a27e-a18dee2aa1af None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:22.306596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338f60d-7a42-48ec-bce2-8871970ce5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:22.307184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338f60d-7a42-48ec-bce2-8871970ce5de None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:22.307461 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338f60d-7a42-48ec-bce2-8871970ce5de None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:22.307729 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338f60d-7a42-48ec-bce2-8871970ce5de None admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:22.308411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338f60d-7a42-48ec-bce2-8871970ce5de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:22.308709 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338f60d-7a42-48ec-bce2-8871970ce5de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:22.310150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338f60d-7a42-48ec-bce2-8871970ce5de None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:22.321477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 647/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Tue Oct 4 08:36:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:22.327535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ac278-6841-4b26-8c31-bd4a8204a1f2 None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:22.423460 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ac278-6841-4b26-8c31-bd4a8204a1f2 None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:22.454707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac0585-6cfa-4416-86a4-447015c2b3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:22.455291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac0585-6cfa-4416-86a4-447015c2b3f4 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:22.455567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac0585-6cfa-4416-86a4-447015c2b3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:22.455833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac0585-6cfa-4416-86a4-447015c2b3f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:22.456531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac0585-6cfa-4416-86a4-447015c2b3f4 None admin] RBAC: Adding query filter params (role.id=f763a168cafd44ffb29843b3716d5f50, scope.project.id=fa3df14a62714f6bab2acdfc979b655b, user.id=abd158648914409f87ce0599843f0fcf) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:22.456835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac0585-6cfa-4416-86a4-447015c2b3f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:22.458270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac0585-6cfa-4416-86a4-447015c2b3f4 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:22.463728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 640/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Tue Oct 4 08:36:22 2022] GET /identity/v3/role_assignments?user.id=abd158648914409f87ce0599843f0fcf&scope.project.id=fa3df14a62714f6bab2acdfc979b655b&role.id=f763a168cafd44ffb29843b3716d5f50 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:24.608753 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82c591-036f-46bd-81fd-74ce3855a571 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:24.609275 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82c591-036f-46bd-81fd-74ce3855a571 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:24.609365 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82c591-036f-46bd-81fd-74ce3855a571 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:24.610653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 648/1288] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:36:24.615692 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4fa44-fea1-460a-87b4-0f1842e4774a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:24.616080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4fa44-fea1-460a-87b4-0f1842e4774a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:24.616239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4fa44-fea1-460a-87b4-0f1842e4774a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:24.634473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-37e4fa44-fea1-460a-87b4-0f1842e4774a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:36:24.635652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-37e4fa44-fea1-460a-87b4-0f1842e4774a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:36:24.664422 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:24.701261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 641/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:36:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:36:24.969209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82c591-036f-46bd-81fd-74ce3855a571 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:25.072190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82c591-036f-46bd-81fd-74ce3855a571 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:25.082312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:25.105482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af2f2-f110-4218-9970-edaf5d7ad16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:25.106119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af2f2-f110-4218-9970-edaf5d7ad16d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:25.106398 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af2f2-f110-4218-9970-edaf5d7ad16d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:25.106665 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565af2f2-f110-4218-9970-edaf5d7ad16d None admin] PATH_INFO: `/v3/services/placement` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:25.107238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565af2f2-f110-4218-9970-edaf5d7ad16d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:25.108587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565af2f2-f110-4218-9970-edaf5d7ad16d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-565af2f2-f110-4218-9970-edaf5d7ad16d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:25.116927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:25.118187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:25.119412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 649/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 08:36:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:25.124497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4fa44-fea1-460a-87b4-0f1842e4774a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:25.216749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4fa44-fea1-460a-87b4-0f1842e4774a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:25.249419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652344c-7d08-42bb-b666-bfa20ddf7b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:25.249972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652344c-7d08-42bb-b666-bfa20ddf7b8a None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:25.250246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652344c-7d08-42bb-b666-bfa20ddf7b8a None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:25.250511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652344c-7d08-42bb-b666-bfa20ddf7b8a None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:25.251119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652344c-7d08-42bb-b666-bfa20ddf7b8a None admin] RBAC: Adding query filter params (name=placement) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:25.251418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652344c-7d08-42bb-b666-bfa20ddf7b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:25.252765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652344c-7d08-42bb-b666-bfa20ddf7b8a None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:25.257265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 642/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Oct 4 08:36:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:25.262774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af2f2-f110-4218-9970-edaf5d7ad16d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:25.365942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565af2f2-f110-4218-9970-edaf5d7ad16d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:25.398797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d3d33-eb31-48bb-b6c8-e03401cdc031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:25.399376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d3d33-eb31-48bb-b6c8-e03401cdc031 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:25.399658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d3d33-eb31-48bb-b6c8-e03401cdc031 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:25.399931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d3d33-eb31-48bb-b6c8-e03401cdc031 None admin] PATH_INFO: `/v3/services` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:25.400552 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d3d33-eb31-48bb-b6c8-e03401cdc031 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:25.400854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d3d33-eb31-48bb-b6c8-e03401cdc031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:25.402265 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d3d33-eb31-48bb-b6c8-e03401cdc031 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:25.406634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 650/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Oct 4 08:36:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:36:27.514815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab280d-3bc7-4ba5-9b03-6cf6c1749665 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:27.515347 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab280d-3bc7-4ba5-9b03-6cf6c1749665 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:27.515347 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab280d-3bc7-4ba5-9b03-6cf6c1749665 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:27.516693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 643/1293] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:36:27.522006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fb8ad-e372-4994-b7ab-2565cba148e5 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:27.522405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fb8ad-e372-4994-b7ab-2565cba148e5 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:27.522571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fb8ad-e372-4994-b7ab-2565cba148e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:27.541004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-4b3fb8ad-e372-4994-b7ab-2565cba148e5 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:36:27.542229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4b3fb8ad-e372-4994-b7ab-2565cba148e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:36:27.571346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:27.608311 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 651/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:36:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:36:27.880890 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab280d-3bc7-4ba5-9b03-6cf6c1749665 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:27.911475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:27.980737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab280d-3bc7-4ba5-9b03-6cf6c1749665 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:28.013249 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ac849-90b9-484b-a473-3528a85b9572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:28.013867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac849-90b9-484b-a473-3528a85b9572 None admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:28.014151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac849-90b9-484b-a473-3528a85b9572 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:28.014423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac849-90b9-484b-a473-3528a85b9572 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:28.015061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ac849-90b9-484b-a473-3528a85b9572 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:28.016436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ac849-90b9-484b-a473-3528a85b9572 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:28.030223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 644/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Tue Oct 4 08:36:27 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:36:30.141353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e2dde-3f15-4275-8d8a-cd5812c603b1 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:30.141906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e2dde-3f15-4275-8d8a-cd5812c603b1 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:30.141906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e2dde-3f15-4275-8d8a-cd5812c603b1 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:30.143258 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 652/1296] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:36:30.148503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eace82-4057-45c7-b51f-0f8d689aa0b3 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:30.148934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eace82-4057-45c7-b51f-0f8d689aa0b3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:30.149060 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eace82-4057-45c7-b51f-0f8d689aa0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:30.166765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-b7eace82-4057-45c7-b51f-0f8d689aa0b3 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:36:30.168482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b7eace82-4057-45c7-b51f-0f8d689aa0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:36:30.237468 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 645/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:36:30 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:36:30.516889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e2dde-3f15-4275-8d8a-cd5812c603b1 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:30.615209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e2dde-3f15-4275-8d8a-cd5812c603b1 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:30.647771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cc9bb-33be-485b-9025-938fc556044d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:30.648384 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cc9bb-33be-485b-9025-938fc556044d None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:30.648663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cc9bb-33be-485b-9025-938fc556044d None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:30.648937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cc9bb-33be-485b-9025-938fc556044d None admin] PATH_INFO: `/v3/services/placement` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:30.649523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6cc9bb-33be-485b-9025-938fc556044d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:30.650928 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6cc9bb-33be-485b-9025-938fc556044d None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-ef6cc9bb-33be-485b-9025-938fc556044d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:30.659996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:30.661184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:30.662218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 653/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 08:36:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:30.666927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eace82-4057-45c7-b51f-0f8d689aa0b3 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:30.685586 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:30.762764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eace82-4057-45c7-b51f-0f8d689aa0b3 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:30.795745 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af1167-dfaf-48cd-b3d8-c4df91709b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:30.796285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af1167-dfaf-48cd-b3d8-c4df91709b4d None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:30.796562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af1167-dfaf-48cd-b3d8-c4df91709b4d None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:30.796833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af1167-dfaf-48cd-b3d8-c4df91709b4d None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:30.797448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af1167-dfaf-48cd-b3d8-c4df91709b4d None admin] RBAC: Adding query filter params (name=placement) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:30.797772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af1167-dfaf-48cd-b3d8-c4df91709b4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:30.799126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af1167-dfaf-48cd-b3d8-c4df91709b4d None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:30.804047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 646/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Oct 4 08:36:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:30.810537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cc9bb-33be-485b-9025-938fc556044d None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:30.840879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:30.912174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cc9bb-33be-485b-9025-938fc556044d None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:30.947947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8f7f5-90ac-4a61-a876-3536023acbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:30.948615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8f7f5-90ac-4a61-a876-3536023acbf0 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:30.948909 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8f7f5-90ac-4a61-a876-3536023acbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:30.949212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8f7f5-90ac-4a61-a876-3536023acbf0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:30.949951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8f7f5-90ac-4a61-a876-3536023acbf0 None admin] RBAC: Adding query filter params (interface=public, service_id=ac7b008f21964e6c9ce1c18e71c1eccc, region_id=RegionOne) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:30.950263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8f7f5-90ac-4a61-a876-3536023acbf0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:30.951712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8f7f5-90ac-4a61-a876-3536023acbf0 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:30.956813 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 654/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Tue Oct 4 08:36:30 2022] GET /identity/v3/endpoints?service_id=ac7b008f21964e6c9ce1c18e71c1eccc&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:30.964978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af1167-dfaf-48cd-b3d8-c4df91709b4d None admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:31.052153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:31.069814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af1167-dfaf-48cd-b3d8-c4df91709b4d None admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:31.102234 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908e79d-14c3-408a-8d2d-9ce76427f8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:31.102828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908e79d-14c3-408a-8d2d-9ce76427f8db None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:31.103105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908e79d-14c3-408a-8d2d-9ce76427f8db None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:31.103374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908e79d-14c3-408a-8d2d-9ce76427f8db None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:31.103969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908e79d-14c3-408a-8d2d-9ce76427f8db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:31.106008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908e79d-14c3-408a-8d2d-9ce76427f8db None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:31.112511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 647/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Tue Oct 4 08:36:30 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:36:33.238317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d6b10-a9a0-4368-a5cf-de02875d4adb None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:33.238866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d6b10-a9a0-4368-a5cf-de02875d4adb None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:33.238866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d6b10-a9a0-4368-a5cf-de02875d4adb None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:33.240179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 655/1302] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:36:33.245370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed5c22-80f3-469d-97ce-d9d485898fbd None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:33.245776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed5c22-80f3-469d-97ce-d9d485898fbd None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:33.245953 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed5c22-80f3-469d-97ce-d9d485898fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:33.264044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-92ed5c22-80f3-469d-97ce-d9d485898fbd None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:36:33.265184 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-92ed5c22-80f3-469d-97ce-d9d485898fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:36:33.330317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 648/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:36:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 08:36:33.601493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d6b10-a9a0-4368-a5cf-de02875d4adb None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:33.695066 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:33.701987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d6b10-a9a0-4368-a5cf-de02875d4adb None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:33.734568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d411635-2bd7-4519-8338-9fdbf54e4a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:33.735181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d411635-2bd7-4519-8338-9fdbf54e4a51 None admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:33.735458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d411635-2bd7-4519-8338-9fdbf54e4a51 None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:33.735725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d411635-2bd7-4519-8338-9fdbf54e4a51 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:33.736291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d411635-2bd7-4519-8338-9fdbf54e4a51 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:33.737638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d411635-2bd7-4519-8338-9fdbf54e4a51 None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application [None req-8d411635-2bd7-4519-8338-9fdbf54e4a51 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 08:36:33.746774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 08:36:33.749892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application with Lock( Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self._enter() Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application return self.creator() Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application created_value = creator( Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: ERROR keystone.server.flask.application Oct 04 08:36:33.751073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 656/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Tue Oct 4 08:36:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 08:36:33.754005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed5c22-80f3-469d-97ce-d9d485898fbd None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:33.827573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:33.854042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed5c22-80f3-469d-97ce-d9d485898fbd None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:33.886280 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df5212-4d66-4dc2-b5bf-1887cb969f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:33.886869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df5212-4d66-4dc2-b5bf-1887cb969f92 None admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:33.887145 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df5212-4d66-4dc2-b5bf-1887cb969f92 None admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:33.887449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df5212-4d66-4dc2-b5bf-1887cb969f92 None admin] PATH_INFO: `/v3/services` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:33.888062 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df5212-4d66-4dc2-b5bf-1887cb969f92 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:36:33.888362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df5212-4d66-4dc2-b5bf-1887cb969f92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:33.889780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df5212-4d66-4dc2-b5bf-1887cb969f92 None admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:33.894704 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 649/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Tue Oct 4 08:36:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:36:33.900413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d411635-2bd7-4519-8338-9fdbf54e4a51 None admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:36:33.996659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d411635-2bd7-4519-8338-9fdbf54e4a51 None admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:36:34.028849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04f5d6-dea0-4dfa-83a8-8442034dd86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:36:34.029423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04f5d6-dea0-4dfa-83a8-8442034dd86c None admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:36:34.029716 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04f5d6-dea0-4dfa-83a8-8442034dd86c None admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:36:34.029994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04f5d6-dea0-4dfa-83a8-8442034dd86c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:36:34.030652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04f5d6-dea0-4dfa-83a8-8442034dd86c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:36:34.031993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04f5d6-dea0-4dfa-83a8-8442034dd86c None admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:36:34.050567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 657/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:36:33 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:36:43.892492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=30 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:36:44.046324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:38.296756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894d66a-e1f9-47c9-8cd9-1aa654a8b0c2 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:38.297236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894d66a-e1f9-47c9-8cd9-1aa654a8b0c2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:38.297340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894d66a-e1f9-47c9-8cd9-1aa654a8b0c2 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:38.304690 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2246f3-5c9b-4bbc-b74d-1d10ba4406b1 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:38.305158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2246f3-5c9b-4bbc-b74d-1d10ba4406b1 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:38.305253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2246f3-5c9b-4bbc-b74d-1d10ba4406b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:38.308311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 650/1307] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:37:38 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:38.408022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-8d2246f3-5c9b-4bbc-b74d-1d10ba4406b1 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:38.409312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8d2246f3-5c9b-4bbc-b74d-1d10ba4406b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:38.517787 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 658/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:37:38.808083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c074dc-9f1f-4a00-b90c-a9068ab80833 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:38.808548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c074dc-9f1f-4a00-b90c-a9068ab80833 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:38.808885 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c074dc-9f1f-4a00-b90c-a9068ab80833 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:38.810475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 651/1309] 10.222.0.26 () {54 vars in 964 bytes} [Tue Oct 4 08:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:38.816742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274dcad-d3a2-4968-b571-184665dc92db None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:38.817031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274dcad-d3a2-4968-b571-184665dc92db None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:38.817659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274dcad-d3a2-4968-b571-184665dc92db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:38.908204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b274dcad-d3a2-4968-b571-184665dc92db None None] MFA Rules not processed for user `50d3b75cbc5c4d1686f4d93248e04746`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:38.909555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b274dcad-d3a2-4968-b571-184665dc92db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:38.932744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:38.996443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 659/1310] 10.222.0.26 () {58 vars in 1061 bytes} [Tue Oct 4 08:37:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:37:39.005842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c074dc-9f1f-4a00-b90c-a9068ab80833 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:39.008279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-77c074dc-9f1f-4a00-b90c-a9068ab80833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:39.132786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c074dc-9f1f-4a00-b90c-a9068ab80833 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:39.214035 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cf3b4-e38d-4a6d-8b0d-297db6cfd239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:39.214561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cf3b4-e38d-4a6d-8b0d-297db6cfd239 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:39.214846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cf3b4-e38d-4a6d-8b0d-297db6cfd239 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:39.215122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cf3b4-e38d-4a6d-8b0d-297db6cfd239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:39.216767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-bb3cf3b4-e38d-4a6d-8b0d-297db6cfd239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:39.277871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:39.337983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cf3b4-e38d-4a6d-8b0d-297db6cfd239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:39.339499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cf3b4-e38d-4a6d-8b0d-297db6cfd239 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:39.359851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 652/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:37:41.856098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17958ef-1578-42c0-9bba-a6319e3cdba0 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:41.856633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17958ef-1578-42c0-9bba-a6319e3cdba0 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:41.856633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17958ef-1578-42c0-9bba-a6319e3cdba0 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:41.858034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 660/1312] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:41.863312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19528599-6273-473f-9166-a89a56036f02 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:41.863719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19528599-6273-473f-9166-a89a56036f02 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:41.863989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19528599-6273-473f-9166-a89a56036f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:41.946090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-19528599-6273-473f-9166-a89a56036f02 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:41.947300 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-19528599-6273-473f-9166-a89a56036f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:42.044093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 653/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:37:42.332849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17958ef-1578-42c0-9bba-a6319e3cdba0 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:42.335034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f17958ef-1578-42c0-9bba-a6319e3cdba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:42.438564 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17958ef-1578-42c0-9bba-a6319e3cdba0 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:42.508883 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:42.511279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2429d90-04c6-4a25-811a-b460830997b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:42.512105 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2429d90-04c6-4a25-811a-b460830997b8 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:42.512563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2429d90-04c6-4a25-811a-b460830997b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:42.513011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2429d90-04c6-4a25-811a-b460830997b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:42.514660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e2429d90-04c6-4a25-811a-b460830997b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:42.620462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2429d90-04c6-4a25-811a-b460830997b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:42.621859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2429d90-04c6-4a25-811a-b460830997b8 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:42.641240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 661/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:37:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:37:45.109351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664902c-f44b-4249-a004-431ad1f30079 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:45.109964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664902c-f44b-4249-a004-431ad1f30079 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:45.109964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664902c-f44b-4249-a004-431ad1f30079 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:45.111372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 654/1315] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:45.117014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c742985-ec04-4eb1-a224-de176db09d4f None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:45.117420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c742985-ec04-4eb1-a224-de176db09d4f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:45.117568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c742985-ec04-4eb1-a224-de176db09d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:45.182307 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-8c742985-ec04-4eb1-a224-de176db09d4f None None] MFA Rules not processed for user `902e1680a1bd4607ad018968c6bfe87c`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:45.183639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8c742985-ec04-4eb1-a224-de176db09d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:45.234315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:45.278801 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 662/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:37:45.560790 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664902c-f44b-4249-a004-431ad1f30079 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:45.562689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2664902c-f44b-4249-a004-431ad1f30079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:45.573140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:45.669402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664902c-f44b-4249-a004-431ad1f30079 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:45.744252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344283c-a26c-488e-82ed-8a71c96f2208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:45.744846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344283c-a26c-488e-82ed-8a71c96f2208 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:45.745121 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344283c-a26c-488e-82ed-8a71c96f2208 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:45.745387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344283c-a26c-488e-82ed-8a71c96f2208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:45.746980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e344283c-a26c-488e-82ed-8a71c96f2208 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:45.815435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:45.863717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344283c-a26c-488e-82ed-8a71c96f2208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:45.865162 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344283c-a26c-488e-82ed-8a71c96f2208 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:45.883360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 655/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:37:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:37:48.599467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd112683-4ccd-4fcf-93e3-904e41b786ae None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:48.600535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd112683-4ccd-4fcf-93e3-904e41b786ae None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:48.600535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd112683-4ccd-4fcf-93e3-904e41b786ae None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:48.601892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 663/1318] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:48.607143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f664c4-72fb-4dc8-867f-a3126e0f2997 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:48.607525 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f664c4-72fb-4dc8-867f-a3126e0f2997 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:48.607702 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f664c4-72fb-4dc8-867f-a3126e0f2997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:48.674699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-48f664c4-72fb-4dc8-867f-a3126e0f2997 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:48.675970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-48f664c4-72fb-4dc8-867f-a3126e0f2997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:48.769233 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 656/1319] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:37:49.051522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd112683-4ccd-4fcf-93e3-904e41b786ae None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:49.053119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-dd112683-4ccd-4fcf-93e3-904e41b786ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:49.152897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd112683-4ccd-4fcf-93e3-904e41b786ae None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:49.224495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cf8c2-5ced-4c27-946d-87309dba4dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:49.225670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cf8c2-5ced-4c27-946d-87309dba4dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:49.226007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cf8c2-5ced-4c27-946d-87309dba4dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:49.228492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cf8c2-5ced-4c27-946d-87309dba4dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:49.228492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7b2cf8c2-5ced-4c27-946d-87309dba4dc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:49.234726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:49.337405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cf8c2-5ced-4c27-946d-87309dba4dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:49.338805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cf8c2-5ced-4c27-946d-87309dba4dc8 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:49.357146 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 664/1320] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:37:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:37:52.409651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3257ea3-ffba-45d2-9fef-eb58b82c8d1a None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:52.410236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3257ea3-ffba-45d2-9fef-eb58b82c8d1a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:52.410345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3257ea3-ffba-45d2-9fef-eb58b82c8d1a None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:52.411767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 657/1321] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:52.417532 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16490deb-7443-4202-9331-863cefcc24b1 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:52.417995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16490deb-7443-4202-9331-863cefcc24b1 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:52.418224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16490deb-7443-4202-9331-863cefcc24b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:52.487956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-16490deb-7443-4202-9331-863cefcc24b1 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:52.489487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-16490deb-7443-4202-9331-863cefcc24b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:52.548321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:52.589769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 665/1322] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:37:52.882307 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3257ea3-ffba-45d2-9fef-eb58b82c8d1a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:52.883892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a3257ea3-ffba-45d2-9fef-eb58b82c8d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:52.903165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:52.986217 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3257ea3-ffba-45d2-9fef-eb58b82c8d1a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:53.058771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fd55d-de6e-443e-8966-49642b09da19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:53.059362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fd55d-de6e-443e-8966-49642b09da19 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:53.059645 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fd55d-de6e-443e-8966-49642b09da19 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:53.059909 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fd55d-de6e-443e-8966-49642b09da19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:53.061480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6f7fd55d-de6e-443e-8966-49642b09da19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:53.131600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:53.170362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fd55d-de6e-443e-8966-49642b09da19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:53.171823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fd55d-de6e-443e-8966-49642b09da19 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:53.189502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 658/1323] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:37:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:37:55.400444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054d7cd-cedc-461b-a318-c6c606474e63 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.400997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054d7cd-cedc-461b-a318-c6c606474e63 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.400997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054d7cd-cedc-461b-a318-c6c606474e63 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.402394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 666/1324] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:37:55.407788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dde793-b761-41e9-97a6-41bd66c6aee5 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.408224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dde793-b761-41e9-97a6-41bd66c6aee5 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.408332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dde793-b761-41e9-97a6-41bd66c6aee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.474823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f4dde793-b761-41e9-97a6-41bd66c6aee5 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:55.476033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f4dde793-b761-41e9-97a6-41bd66c6aee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:55.569854 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 659/1325] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:37:55.863993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ec44c-75bd-4c1e-83e3-f1532ad7bbf2 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.864284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ec44c-75bd-4c1e-83e3-f1532ad7bbf2 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.864556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ec44c-75bd-4c1e-83e3-f1532ad7bbf2 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.866434 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 667/1326] 10.222.0.26 () {54 vars in 964 bytes} [Tue Oct 4 08:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:55.872456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f8926-4c03-4f06-b91a-c9e6807ba552 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:55.872820 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f8926-4c03-4f06-b91a-c9e6807ba552 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:55.873168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f8926-4c03-4f06-b91a-c9e6807ba552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:55.906003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:55.952527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-9a2f8926-4c03-4f06-b91a-c9e6807ba552 None None] MFA Rules not processed for user `50d3b75cbc5c4d1686f4d93248e04746`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:55.953830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9a2f8926-4c03-4f06-b91a-c9e6807ba552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:56.031209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 660/1327] 10.222.0.26 () {58 vars in 1061 bytes} [Tue Oct 4 08:37:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:37:56.040668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ec44c-75bd-4c1e-83e3-f1532ad7bbf2 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:56.042213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-dc4ec44c-75bd-4c1e-83e3-f1532ad7bbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:56.143530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ec44c-75bd-4c1e-83e3-f1532ad7bbf2 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:56.216988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32ced9-4655-4873-8c53-e6f6ddbecc43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:56.217587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32ced9-4655-4873-8c53-e6f6ddbecc43 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:56.217892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32ced9-4655-4873-8c53-e6f6ddbecc43 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:56.218164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32ced9-4655-4873-8c53-e6f6ddbecc43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:56.219744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8d32ced9-4655-4873-8c53-e6f6ddbecc43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:56.237439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:56.329060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32ced9-4655-4873-8c53-e6f6ddbecc43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:56.330540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32ced9-4655-4873-8c53-e6f6ddbecc43 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:56.350279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 668/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:37:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:37:58.763773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c318c27-2f69-4bf3-9df2-6adcfc4d06c3 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:58.764246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c318c27-2f69-4bf3-9df2-6adcfc4d06c3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:58.764246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c318c27-2f69-4bf3-9df2-6adcfc4d06c3 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:58.765576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 661/1329] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:37:58.770721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77cf7e-82f2-4282-9992-f62f492f4449 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:58.771141 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77cf7e-82f2-4282-9992-f62f492f4449 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:58.771301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77cf7e-82f2-4282-9992-f62f492f4449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:58.837926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-2b77cf7e-82f2-4282-9992-f62f492f4449 None None] MFA Rules not processed for user `902e1680a1bd4607ad018968c6bfe87c`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:37:58.839131 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2b77cf7e-82f2-4282-9992-f62f492f4449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:58.903652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:58.935446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 669/1330] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:37:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:37:59.218345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c318c27-2f69-4bf3-9df2-6adcfc4d06c3 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:37:59.219879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3c318c27-2f69-4bf3-9df2-6adcfc4d06c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:59.330026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c318c27-2f69-4bf3-9df2-6adcfc4d06c3 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:37:59.351125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:37:59.406660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e5ed9-2d7b-4ea6-82fa-a89c56cc1fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:37:59.407269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e5ed9-2d7b-4ea6-82fa-a89c56cc1fdb service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:37:59.407548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e5ed9-2d7b-4ea6-82fa-a89c56cc1fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:37:59.407818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e5ed9-2d7b-4ea6-82fa-a89c56cc1fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:37:59.409417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-815e5ed9-2d7b-4ea6-82fa-a89c56cc1fdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:37:59.538983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e5ed9-2d7b-4ea6-82fa-a89c56cc1fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:37:59.540606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e5ed9-2d7b-4ea6-82fa-a89c56cc1fdb service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:37:59.562358 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 662/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:37:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:02.088108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e682f-a97f-45ea-a21f-1eaca8089777 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:02.088699 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e682f-a97f-45ea-a21f-1eaca8089777 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:02.088699 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e682f-a97f-45ea-a21f-1eaca8089777 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:02.090034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 670/1332] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:02.095316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de270fa9-fb9f-45e0-934d-0791e874fdb2 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:02.095733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de270fa9-fb9f-45e0-934d-0791e874fdb2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:02.095899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de270fa9-fb9f-45e0-934d-0791e874fdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:02.158247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:02.160945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-de270fa9-fb9f-45e0-934d-0791e874fdb2 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:02.162769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-de270fa9-fb9f-45e0-934d-0791e874fdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:02.257514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 663/1333] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:02.539112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e682f-a97f-45ea-a21f-1eaca8089777 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:02.540669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f14e682f-a97f-45ea-a21f-1eaca8089777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:02.640997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e682f-a97f-45ea-a21f-1eaca8089777 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:02.717128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e55490-66d3-4324-976b-d45bcd974f68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:02.717748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e55490-66d3-4324-976b-d45bcd974f68 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:02.718026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e55490-66d3-4324-976b-d45bcd974f68 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:02.718293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e55490-66d3-4324-976b-d45bcd974f68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:02.719876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-84e55490-66d3-4324-976b-d45bcd974f68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:02.746932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:02.827382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e55490-66d3-4324-976b-d45bcd974f68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:02.828743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e55490-66d3-4324-976b-d45bcd974f68 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:02.846193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 671/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:05.025190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415e6d0-15b0-4b57-8403-552850e5689a None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:05.025738 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415e6d0-15b0-4b57-8403-552850e5689a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:05.025859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415e6d0-15b0-4b57-8403-552850e5689a None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:05.027110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 664/1335] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:05.032243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38854e-95cb-4f79-942b-622c59d59a17 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:05.032735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38854e-95cb-4f79-942b-622c59d59a17 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:05.032833 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38854e-95cb-4f79-942b-622c59d59a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:05.098533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6f38854e-95cb-4f79-942b-622c59d59a17 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:05.099829 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6f38854e-95cb-4f79-942b-622c59d59a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:05.176216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:05.198853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 672/1336] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:05.487190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415e6d0-15b0-4b57-8403-552850e5689a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:05.488734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e415e6d0-15b0-4b57-8403-552850e5689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:05.590200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415e6d0-15b0-4b57-8403-552850e5689a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:05.619354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:05.664266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631947-9634-4cf3-ad78-8bbf40a5fb6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:05.664874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631947-9634-4cf3-ad78-8bbf40a5fb6e service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:05.665152 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631947-9634-4cf3-ad78-8bbf40a5fb6e service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:05.665420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631947-9634-4cf3-ad78-8bbf40a5fb6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:05.667061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9d631947-9634-4cf3-ad78-8bbf40a5fb6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:05.773266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d631947-9634-4cf3-ad78-8bbf40a5fb6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:05.774721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d631947-9634-4cf3-ad78-8bbf40a5fb6e service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:05.792501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 665/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:08.325009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e82c37-7360-477c-b252-84ed9c37dadf None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:08.325643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e82c37-7360-477c-b252-84ed9c37dadf None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:08.325643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e82c37-7360-477c-b252-84ed9c37dadf None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:08.327005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 673/1338] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:08.333272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760710f-eba8-4508-9dd5-535547784b9e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:08.333725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760710f-eba8-4508-9dd5-535547784b9e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:08.333909 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760710f-eba8-4508-9dd5-535547784b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:08.401436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-4760710f-eba8-4508-9dd5-535547784b9e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:08.402694 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4760710f-eba8-4508-9dd5-535547784b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:08.410993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:08.499176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 666/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:08.797053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e82c37-7360-477c-b252-84ed9c37dadf None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:08.798672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-83e82c37-7360-477c-b252-84ed9c37dadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:08.900029 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e82c37-7360-477c-b252-84ed9c37dadf None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:08.971773 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788b1ea-674e-4e86-a7ab-1c75a39f8688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:08.972312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788b1ea-674e-4e86-a7ab-1c75a39f8688 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:08.972591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788b1ea-674e-4e86-a7ab-1c75a39f8688 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:08.972862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788b1ea-674e-4e86-a7ab-1c75a39f8688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:08.974981 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2788b1ea-674e-4e86-a7ab-1c75a39f8688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:09.009357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:09.080886 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2788b1ea-674e-4e86-a7ab-1c75a39f8688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:09.082342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2788b1ea-674e-4e86-a7ab-1c75a39f8688 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:09.100422 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 674/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:13.171434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ba89b-8068-4f90-9c4b-fd4d131e2fff None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:13.171991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ba89b-8068-4f90-9c4b-fd4d131e2fff None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:13.171991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ba89b-8068-4f90-9c4b-fd4d131e2fff None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:13.173231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 667/1341] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:13.179144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a913e3-77b1-4482-a09d-a994d23b3939 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:13.179563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a913e3-77b1-4482-a09d-a994d23b3939 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:13.179774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a913e3-77b1-4482-a09d-a994d23b3939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:13.246560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-23a913e3-77b1-4482-a09d-a994d23b3939 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:13.247660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-23a913e3-77b1-4482-a09d-a994d23b3939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:13.335762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:13.348529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 675/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:13.633195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ba89b-8068-4f90-9c4b-fd4d131e2fff None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:13.634819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9b8ba89b-8068-4f90-9c4b-fd4d131e2fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:13.740692 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ba89b-8068-4f90-9c4b-fd4d131e2fff None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:13.778985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:13.815638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e84e72-82b0-490a-befb-17794170d091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:13.816251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e84e72-82b0-490a-befb-17794170d091 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:13.816525 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e84e72-82b0-490a-befb-17794170d091 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:13.817393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e84e72-82b0-490a-befb-17794170d091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:13.819029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-94e84e72-82b0-490a-befb-17794170d091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:13.927616 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e84e72-82b0-490a-befb-17794170d091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:13.928969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e84e72-82b0-490a-befb-17794170d091 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:13.946862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 668/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:16.767157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7ea6f-a422-45b2-aac6-4d9927ff849c None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:16.767728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7ea6f-a422-45b2-aac6-4d9927ff849c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:16.767728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7ea6f-a422-45b2-aac6-4d9927ff849c None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:16.769036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 676/1344] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:16.774263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1a3dd-07b8-4479-8859-525aff850fcc None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:16.774692 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1a3dd-07b8-4479-8859-525aff850fcc None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:16.774834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1a3dd-07b8-4479-8859-525aff850fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:16.839308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-9fe1a3dd-07b8-4479-8859-525aff850fcc None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:16.840491 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9fe1a3dd-07b8-4479-8859-525aff850fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:16.858356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:16.937225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 669/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:17.230095 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7ea6f-a422-45b2-aac6-4d9927ff849c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:17.231638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9cd7ea6f-a422-45b2-aac6-4d9927ff849c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:17.349300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7ea6f-a422-45b2-aac6-4d9927ff849c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:17.422141 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b141273-706e-4170-8582-0eaa933bde77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:17.422741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b141273-706e-4170-8582-0eaa933bde77 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:17.423016 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b141273-706e-4170-8582-0eaa933bde77 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:17.423281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b141273-706e-4170-8582-0eaa933bde77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:17.424870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8b141273-706e-4170-8582-0eaa933bde77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:17.468155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:17.536425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b141273-706e-4170-8582-0eaa933bde77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:17.537908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b141273-706e-4170-8582-0eaa933bde77 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:17.556422 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 677/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:21.685469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407eb3a2-9569-4a84-8400-60d1906cddaf None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.685974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407eb3a2-9569-4a84-8400-60d1906cddaf None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.686178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407eb3a2-9569-4a84-8400-60d1906cddaf None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.687432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 670/1347] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:21.693022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc755d0d-3047-45bd-adc2-88e346951c18 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:21.693443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc755d0d-3047-45bd-adc2-88e346951c18 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:21.693652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc755d0d-3047-45bd-adc2-88e346951c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:21.774396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-bc755d0d-3047-45bd-adc2-88e346951c18 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:21.775636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-bc755d0d-3047-45bd-adc2-88e346951c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:21.881869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:21.882129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 678/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:22.171215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407eb3a2-9569-4a84-8400-60d1906cddaf None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:22.172792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-407eb3a2-9569-4a84-8400-60d1906cddaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:22.274640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407eb3a2-9569-4a84-8400-60d1906cddaf None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:22.322427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:22.348627 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f597c6d-05ca-4d99-8b87-31eb6b2932f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:22.349222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f597c6d-05ca-4d99-8b87-31eb6b2932f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:22.349502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f597c6d-05ca-4d99-8b87-31eb6b2932f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:22.349811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f597c6d-05ca-4d99-8b87-31eb6b2932f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:22.351398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4f597c6d-05ca-4d99-8b87-31eb6b2932f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:22.460022 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f597c6d-05ca-4d99-8b87-31eb6b2932f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:22.461380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f597c6d-05ca-4d99-8b87-31eb6b2932f7 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:22.480168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 671/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:27.395462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97e4ea-30d7-432c-a951-715ff38b7a52 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:27.396538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97e4ea-30d7-432c-a951-715ff38b7a52 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:27.397046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97e4ea-30d7-432c-a951-715ff38b7a52 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:27.399074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 679/1350] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:38:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:27.406640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd5bc5-aab6-4954-9b02-da7a3956fd36 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:27.407078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd5bc5-aab6-4954-9b02-da7a3956fd36 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:27.407270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd5bc5-aab6-4954-9b02-da7a3956fd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:27.502460 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-12bd5bc5-aab6-4954-9b02-da7a3956fd36 None None] MFA Rules not processed for user `abd158648914409f87ce0599843f0fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:27.504261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-12bd5bc5-aab6-4954-9b02-da7a3956fd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:27.542192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:27.708833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 672/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:38:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:38:31.970633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8fc24-601a-4c64-ba93-7f6a9c7f286f None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:31.971405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8fc24-601a-4c64-ba93-7f6a9c7f286f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:31.971703 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8fc24-601a-4c64-ba93-7f6a9c7f286f None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:31.973885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 680/1352] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:31.979303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bece5e-3317-4280-b0cd-e38e4345ae6a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:31.979732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bece5e-3317-4280-b0cd-e38e4345ae6a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:31.980200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bece5e-3317-4280-b0cd-e38e4345ae6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:32.058354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-77bece5e-3317-4280-b0cd-e38e4345ae6a None None] MFA Rules not processed for user `abd158648914409f87ce0599843f0fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:32.059706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-77bece5e-3317-4280-b0cd-e38e4345ae6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:32.135585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:32.174690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 673/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:38:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:33.590041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186a3d7-5885-4680-8489-5bfb6aff6ef8 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:33.590577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186a3d7-5885-4680-8489-5bfb6aff6ef8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:33.590577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186a3d7-5885-4680-8489-5bfb6aff6ef8 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:33.592050 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 681/1354] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:33.597663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4f2b6-4218-494d-8dab-751b915c343a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:33.598140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4f2b6-4218-494d-8dab-751b915c343a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:33.598238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4f2b6-4218-494d-8dab-751b915c343a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:33.684197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f7f4f2b6-4218-494d-8dab-751b915c343a None None] MFA Rules not processed for user `abd158648914409f87ce0599843f0fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:33.685455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f7f4f2b6-4218-494d-8dab-751b915c343a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:33.794461 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 674/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:38:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:38:37.130971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd86b0-3a23-4859-821e-3870247a9a06 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.131491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd86b0-3a23-4859-821e-3870247a9a06 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.131598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd86b0-3a23-4859-821e-3870247a9a06 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.136547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 682/1356] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:37.140167 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935ba40-40f6-499b-8a5d-5f394b6dde83 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.141025 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935ba40-40f6-499b-8a5d-5f394b6dde83 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.141486 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6935ba40-40f6-499b-8a5d-5f394b6dde83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.181238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:37.216028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-6935ba40-40f6-499b-8a5d-5f394b6dde83 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:37.217223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6935ba40-40f6-499b-8a5d-5f394b6dde83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:37.320852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 675/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:37.623178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77132e-eae1-413a-8c16-3a32d8570bd6 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.623482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77132e-eae1-413a-8c16-3a32d8570bd6 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.623875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77132e-eae1-413a-8c16-3a32d8570bd6 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.625272 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 683/1358] 10.222.0.26 () {54 vars in 961 bytes} [Tue Oct 4 08:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:37.636706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61321ce6-2af9-416d-aeaa-96726989e01c None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:37.637282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61321ce6-2af9-416d-aeaa-96726989e01c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:37.638633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61321ce6-2af9-416d-aeaa-96726989e01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:37.748576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-61321ce6-2af9-416d-aeaa-96726989e01c None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:37.750227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-61321ce6-2af9-416d-aeaa-96726989e01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:37.806846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:37.882271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 676/1359] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Oct 4 08:38:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:37.891457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77132e-eae1-413a-8c16-3a32d8570bd6 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:37.892979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-0d77132e-eae1-413a-8c16-3a32d8570bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:38.058141 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77132e-eae1-413a-8c16-3a32d8570bd6 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:38.140781 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:38.187297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a90e9-809f-4b25-bf14-badeaf7d3f80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:38.188164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a90e9-809f-4b25-bf14-badeaf7d3f80 service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:38.188544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a90e9-809f-4b25-bf14-badeaf7d3f80 service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:38.188910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a90e9-809f-4b25-bf14-badeaf7d3f80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:38.191485 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-912a90e9-809f-4b25-bf14-badeaf7d3f80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:38.306039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a90e9-809f-4b25-bf14-badeaf7d3f80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:38.308430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a90e9-809f-4b25-bf14-badeaf7d3f80 service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:38.337650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 684/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:40.742910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da6626-ce50-4b16-b0aa-caf800125aea None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:40.743396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da6626-ce50-4b16-b0aa-caf800125aea None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:40.743396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da6626-ce50-4b16-b0aa-caf800125aea None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:40.745089 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 677/1361] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:40.753645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e775a2b-71e0-4d33-9d0e-39958cafcbe4 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:40.754185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e775a2b-71e0-4d33-9d0e-39958cafcbe4 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:40.754331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e775a2b-71e0-4d33-9d0e-39958cafcbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:40.825575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6e775a2b-71e0-4d33-9d0e-39958cafcbe4 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:40.826847 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6e775a2b-71e0-4d33-9d0e-39958cafcbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:40.869456 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:40.934003 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 685/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:41.255497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da6626-ce50-4b16-b0aa-caf800125aea None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:41.257016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-76da6626-ce50-4b16-b0aa-caf800125aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:41.438420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da6626-ce50-4b16-b0aa-caf800125aea None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:41.447552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:41.568365 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad0883-7a21-4693-a86c-ededd0f7708d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:41.568979 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad0883-7a21-4693-a86c-ededd0f7708d service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:41.569262 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad0883-7a21-4693-a86c-ededd0f7708d service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:41.569533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad0883-7a21-4693-a86c-ededd0f7708d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:41.571168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-58ad0883-7a21-4693-a86c-ededd0f7708d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:41.663091 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad0883-7a21-4693-a86c-ededd0f7708d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:41.664914 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad0883-7a21-4693-a86c-ededd0f7708d service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:41.684524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 678/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:44.204428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea30c3e-67e6-4632-98f1-05fbde3ec470 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.205667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea30c3e-67e6-4632-98f1-05fbde3ec470 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.206031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea30c3e-67e6-4632-98f1-05fbde3ec470 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.207870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 686/1364] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:44.213473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d4e06-f95b-4254-b2b7-991bff1abd7e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.214064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d4e06-f95b-4254-b2b7-991bff1abd7e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.214357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d4e06-f95b-4254-b2b7-991bff1abd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.232178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:44.288543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a80d4e06-f95b-4254-b2b7-991bff1abd7e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:44.290054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a80d4e06-f95b-4254-b2b7-991bff1abd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:44.406203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 679/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:44.696881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83b34-7efd-41cd-9e52-90ab267b2723 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.697337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83b34-7efd-41cd-9e52-90ab267b2723 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.697698 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83b34-7efd-41cd-9e52-90ab267b2723 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.699749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 687/1366] 10.222.0.26 () {54 vars in 961 bytes} [Tue Oct 4 08:38:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:44.705868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ba59b-b7d8-4296-95a9-f8bfcf2121cc None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:44.706255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ba59b-b7d8-4296-95a9-f8bfcf2121cc None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:44.706798 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ba59b-b7d8-4296-95a9-f8bfcf2121cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:44.789519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a98ba59b-b7d8-4296-95a9-f8bfcf2121cc None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:44.790855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a98ba59b-b7d8-4296-95a9-f8bfcf2121cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:44.827986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:44.916257 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 680/1367] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Oct 4 08:38:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:44.924162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b83b34-7efd-41cd-9e52-90ab267b2723 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:44.926328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-96b83b34-7efd-41cd-9e52-90ab267b2723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:45.088122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b83b34-7efd-41cd-9e52-90ab267b2723 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:45.110723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:45.213244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9714d2-5309-4892-971e-f63071360ea4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:45.214195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9714d2-5309-4892-971e-f63071360ea4 service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.214314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9714d2-5309-4892-971e-f63071360ea4 service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.214667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9714d2-5309-4892-971e-f63071360ea4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.216435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-de9714d2-5309-4892-971e-f63071360ea4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:45.320078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9714d2-5309-4892-971e-f63071360ea4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:45.321901 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9714d2-5309-4892-971e-f63071360ea4 service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:45.341632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 688/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:45.403539 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e34b1-1a96-4aaf-80fe-b69d41138a16 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.404065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e34b1-1a96-4aaf-80fe-b69d41138a16 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.404444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e34b1-1a96-4aaf-80fe-b69d41138a16 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.406137 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 681/1369] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:45.411772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd77fc6-2402-49c7-8529-079dc522b85e None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.412058 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd77fc6-2402-49c7-8529-079dc522b85e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.412332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd77fc6-2402-49c7-8529-079dc522b85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.447143 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:45.495802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-dbd77fc6-2402-49c7-8529-079dc522b85e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:45.497479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-dbd77fc6-2402-49c7-8529-079dc522b85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:45.623946 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 689/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:45.953376 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0df6f6-ddfb-495d-a4b1-08c6b446c836 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.953668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0df6f6-ddfb-495d-a4b1-08c6b446c836 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.953968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0df6f6-ddfb-495d-a4b1-08c6b446c836 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:45.956861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 682/1371] 10.222.0.26 () {54 vars in 963 bytes} [Tue Oct 4 08:38:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:45.961118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5ec2e-ab63-48ac-b203-4c3418998749 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:45.961420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5ec2e-ab63-48ac-b203-4c3418998749 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:45.961713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5ec2e-ab63-48ac-b203-4c3418998749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:46.047681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-aff5ec2e-ab63-48ac-b203-4c3418998749 None None] MFA Rules not processed for user `6f224ed840244faea117fe97c29c26d1`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:46.048703 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-aff5ec2e-ab63-48ac-b203-4c3418998749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:46.102667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:46.136120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 690/1372] 10.222.0.26 () {58 vars in 1060 bytes} [Tue Oct 4 08:38:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:46.143700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0df6f6-ddfb-495d-a4b1-08c6b446c836 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:46.145227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5b0df6f6-ddfb-495d-a4b1-08c6b446c836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:46.256787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0df6f6-ddfb-495d-a4b1-08c6b446c836 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:46.303482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:46.341569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea50244-5f8f-4e46-a2b4-6ae67e935e02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:46.342241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea50244-5f8f-4e46-a2b4-6ae67e935e02 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:46.342530 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea50244-5f8f-4e46-a2b4-6ae67e935e02 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:46.342779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea50244-5f8f-4e46-a2b4-6ae67e935e02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:46.344376 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-aea50244-5f8f-4e46-a2b4-6ae67e935e02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:46.475650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea50244-5f8f-4e46-a2b4-6ae67e935e02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:46.477524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea50244-5f8f-4e46-a2b4-6ae67e935e02 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:46.503586 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 683/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:47.785770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d6cdc-00b3-480c-8c40-641b51ab8b66 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:47.786302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d6cdc-00b3-480c-8c40-641b51ab8b66 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:47.786427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d6cdc-00b3-480c-8c40-641b51ab8b66 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:47.788049 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 691/1374] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:47.794253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45666b-cae9-476a-aa8c-a75068f2eb14 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:47.794638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45666b-cae9-476a-aa8c-a75068f2eb14 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:47.794817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45666b-cae9-476a-aa8c-a75068f2eb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:47.862558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-4e45666b-cae9-476a-aa8c-a75068f2eb14 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:47.863769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4e45666b-cae9-476a-aa8c-a75068f2eb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:47.884780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:47.965238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 684/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:38:48.290321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d6cdc-00b3-480c-8c40-641b51ab8b66 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:48.292198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-180d6cdc-00b3-480c-8c40-641b51ab8b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:48.453147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d6cdc-00b3-480c-8c40-641b51ab8b66 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:48.509069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:48.584372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724040b-c788-4775-9500-d95b2949afec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:48.585093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724040b-c788-4775-9500-d95b2949afec service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:48.585375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724040b-c788-4775-9500-d95b2949afec service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:48.585645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724040b-c788-4775-9500-d95b2949afec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:48.587264 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f724040b-c788-4775-9500-d95b2949afec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:48.682740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724040b-c788-4775-9500-d95b2949afec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:48.684404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724040b-c788-4775-9500-d95b2949afec service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:48.704123 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 692/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:50.950608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d8d74-fec8-4c7a-92ca-5f2d8e96810d None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.951147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d8d74-fec8-4c7a-92ca-5f2d8e96810d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.951147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d8d74-fec8-4c7a-92ca-5f2d8e96810d None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:50.952552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 685/1377] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:50.957679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e793ea-7486-4b6b-89fe-f849054dfd84 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:50.958143 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e793ea-7486-4b6b-89fe-f849054dfd84 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:50.958236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e793ea-7486-4b6b-89fe-f849054dfd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:51.025497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b7e793ea-7486-4b6b-89fe-f849054dfd84 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:51.026728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b7e793ea-7486-4b6b-89fe-f849054dfd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:51.034007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:51.133738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 693/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:51.442264 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d8d74-fec8-4c7a-92ca-5f2d8e96810d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:51.443778 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-730d8d74-fec8-4c7a-92ca-5f2d8e96810d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:51.601585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d8d74-fec8-4c7a-92ca-5f2d8e96810d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:51.605771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:51.739431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be2d66-50a2-47ac-8b35-b22a53664850 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:51.740330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be2d66-50a2-47ac-8b35-b22a53664850 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:51.740742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be2d66-50a2-47ac-8b35-b22a53664850 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:51.741173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be2d66-50a2-47ac-8b35-b22a53664850 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:51.743168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-56be2d66-50a2-47ac-8b35-b22a53664850 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:51.846050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be2d66-50a2-47ac-8b35-b22a53664850 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:51.847727 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be2d66-50a2-47ac-8b35-b22a53664850 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:51.869101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 686/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:54.173314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6d6b4-569a-40c7-8024-a0f0528002f2 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:54.173943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6d6b4-569a-40c7-8024-a0f0528002f2 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:54.174610 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6d6b4-569a-40c7-8024-a0f0528002f2 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:54.176053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 694/1380] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:54.182312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f553e8a-0e02-4bd7-88df-67f287035508 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:54.182790 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f553e8a-0e02-4bd7-88df-67f287035508 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:54.182883 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f553e8a-0e02-4bd7-88df-67f287035508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:54.196525 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:54.258532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-9f553e8a-0e02-4bd7-88df-67f287035508 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:54.259748 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9f553e8a-0e02-4bd7-88df-67f287035508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:54.360536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 687/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:54.650571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6d6b4-569a-40c7-8024-a0f0528002f2 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:54.652126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-0cc6d6b4-569a-40c7-8024-a0f0528002f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:54.805444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:54.817278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6d6b4-569a-40c7-8024-a0f0528002f2 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:54.941069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801fc13-1f64-4f70-9d5f-1165b9bd9c0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:54.941859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801fc13-1f64-4f70-9d5f-1165b9bd9c0c service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:54.942141 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801fc13-1f64-4f70-9d5f-1165b9bd9c0c service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:54.942530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801fc13-1f64-4f70-9d5f-1165b9bd9c0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:54.944505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a801fc13-1f64-4f70-9d5f-1165b9bd9c0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:55.049080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801fc13-1f64-4f70-9d5f-1165b9bd9c0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:55.050752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801fc13-1f64-4f70-9d5f-1165b9bd9c0c service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:55.061131 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:55.069954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 695/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:38:57.255117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf4a90-b14a-4432-8901-9cef016edc5e None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:57.255662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf4a90-b14a-4432-8901-9cef016edc5e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:57.255662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf4a90-b14a-4432-8901-9cef016edc5e None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:57.257019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 688/1383] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:38:57.262697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772e76d-7b41-4b89-bf43-8d0ddefbc16c None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:57.263089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772e76d-7b41-4b89-bf43-8d0ddefbc16c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:57.263423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772e76d-7b41-4b89-bf43-8d0ddefbc16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:57.349850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6772e76d-7b41-4b89-bf43-8d0ddefbc16c None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:38:57.351389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6772e76d-7b41-4b89-bf43-8d0ddefbc16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:57.468450 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 696/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:38:57.764271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf4a90-b14a-4432-8901-9cef016edc5e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:38:57.765869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-92cf4a90-b14a-4432-8901-9cef016edc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:57.852320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:57.913759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf4a90-b14a-4432-8901-9cef016edc5e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:38:58.036768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfea79a-6ae0-4479-9997-0c28cc6723f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:38:58.037845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfea79a-6ae0-4479-9997-0c28cc6723f2 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:38:58.038550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfea79a-6ae0-4479-9997-0c28cc6723f2 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:38:58.038900 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfea79a-6ae0-4479-9997-0c28cc6723f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:38:58.040991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ddfea79a-6ae0-4479-9997-0c28cc6723f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:38:58.095192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:38:58.140150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfea79a-6ae0-4479-9997-0c28cc6723f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:38:58.142355 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfea79a-6ae0-4479-9997-0c28cc6723f2 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:38:58.164805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 689/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:38:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:00.437736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99854b6d-faed-493d-a696-c2a1f3bacf79 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.438254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99854b6d-faed-493d-a696-c2a1f3bacf79 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.438254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99854b6d-faed-493d-a696-c2a1f3bacf79 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.440335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 697/1386] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:39:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:00.445580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ea9ae-4796-4e7d-8dd4-013217dc7b4e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:00.446016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ea9ae-4796-4e7d-8dd4-013217dc7b4e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:00.446165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ea9ae-4796-4e7d-8dd4-013217dc7b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:00.511954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d62ea9ae-4796-4e7d-8dd4-013217dc7b4e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:00.513156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d62ea9ae-4796-4e7d-8dd4-013217dc7b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:00.618470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 690/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:00.913236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99854b6d-faed-493d-a696-c2a1f3bacf79 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:00.914847 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-99854b6d-faed-493d-a696-c2a1f3bacf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:00.990954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:01.059598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99854b6d-faed-493d-a696-c2a1f3bacf79 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:01.171957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606f28c-d76a-43a6-8353-6b5fabe87966 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:01.172479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606f28c-d76a-43a6-8353-6b5fabe87966 service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:01.172756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606f28c-d76a-43a6-8353-6b5fabe87966 service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:01.173077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606f28c-d76a-43a6-8353-6b5fabe87966 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:01.174693 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f606f28c-d76a-43a6-8353-6b5fabe87966 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:01.215433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:01.267392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606f28c-d76a-43a6-8353-6b5fabe87966 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:01.269009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606f28c-d76a-43a6-8353-6b5fabe87966 service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:01.287879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 698/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:02.115583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50911b04-db85-40e1-85ba-ddf5daa6f1e0 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.116156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50911b04-db85-40e1-85ba-ddf5daa6f1e0 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.116265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50911b04-db85-40e1-85ba-ddf5daa6f1e0 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.117588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 691/1389] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:02.122919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b2bdb-186e-46b5-9ba6-b7ab1e7171a0 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.123329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b2bdb-186e-46b5-9ba6-b7ab1e7171a0 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.123497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b2bdb-186e-46b5-9ba6-b7ab1e7171a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.191097 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b83b2bdb-186e-46b5-9ba6-b7ab1e7171a0 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:02.192348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b83b2bdb-186e-46b5-9ba6-b7ab1e7171a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:02.291041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 699/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:02.587795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42c859-ac61-46a4-bc1a-916ae18c8bee None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.588100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42c859-ac61-46a4-bc1a-916ae18c8bee None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.588373 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42c859-ac61-46a4-bc1a-916ae18c8bee None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.589753 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 692/1391] 10.222.0.26 () {54 vars in 962 bytes} [Tue Oct 4 08:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:02.595370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca4035-528c-4dc3-8412-125ec527a9f0 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.595657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca4035-528c-4dc3-8412-125ec527a9f0 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.595930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca4035-528c-4dc3-8412-125ec527a9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.612273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:02.677110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-75ca4035-528c-4dc3-8412-125ec527a9f0 None None] MFA Rules not processed for user `913a866ddc074b7f9aa142c70da98a69`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:02.678792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-75ca4035-528c-4dc3-8412-125ec527a9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:02.764505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 700/1392] 10.222.0.26 () {58 vars in 1059 bytes} [Tue Oct 4 08:39:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:02.773620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42c859-ac61-46a4-bc1a-916ae18c8bee None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:02.775135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5f42c859-ac61-46a4-bc1a-916ae18c8bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:02.806346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:02.879063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42c859-ac61-46a4-bc1a-916ae18c8bee None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:02.951893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc808f-6b3b-486b-a780-3af33fefb8fd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '913a866ddc074b7f9aa142c70da98a69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:02.952494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc808f-6b3b-486b-a780-3af33fefb8fd service glance] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:02.952768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc808f-6b3b-486b-a780-3af33fefb8fd service glance] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:02.953037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc808f-6b3b-486b-a780-3af33fefb8fd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:02.954736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-48fc808f-6b3b-486b-a780-3af33fefb8fd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:03.035000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:03.067092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc808f-6b3b-486b-a780-3af33fefb8fd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:03.068552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc808f-6b3b-486b-a780-3af33fefb8fd service glance] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:03.086255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 693/1393] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:39:03.470974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc871e2e-e350-46ce-b0f8-920c8ae30ba6 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:03.471421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc871e2e-e350-46ce-b0f8-920c8ae30ba6 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:03.471421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc871e2e-e350-46ce-b0f8-920c8ae30ba6 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:03.474052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 701/1394] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:39:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:03.478072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd5aed-b12d-4d95-a31b-1f251aa98784 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:03.478469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd5aed-b12d-4d95-a31b-1f251aa98784 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:03.478604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd5aed-b12d-4d95-a31b-1f251aa98784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:03.546213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-09dd5aed-b12d-4d95-a31b-1f251aa98784 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:03.547425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-09dd5aed-b12d-4d95-a31b-1f251aa98784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:03.648354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 694/1395] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:03.832504 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996d129-d1c9-407b-a2fa-05048fc56089 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:03.832798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996d129-d1c9-407b-a2fa-05048fc56089 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:03.833064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996d129-d1c9-407b-a2fa-05048fc56089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:03.900182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-4996d129-d1c9-407b-a2fa-05048fc56089 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:03.901388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4996d129-d1c9-407b-a2fa-05048fc56089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:03.939122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd5aed-b12d-4d95-a31b-1f251aa98784 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:03.940333 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-09dd5aed-b12d-4d95-a31b-1f251aa98784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:03.958445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:03.982242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:03.991864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 702/1396] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:39:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:04.034632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b6d9-1de8-4deb-9849-96af9438b6bc None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.034920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b6d9-1de8-4deb-9849-96af9438b6bc None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.035196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0b6d9-1de8-4deb-9849-96af9438b6bc None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.036558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 703/1397] 10.222.0.26 () {54 vars in 943 bytes} [Tue Oct 4 08:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:39:04.042755 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53c2fe-0750-45f7-b3c4-b3d13aa631c6 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.043042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53c2fe-0750-45f7-b3c4-b3d13aa631c6 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.043313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53c2fe-0750-45f7-b3c4-b3d13aa631c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.059260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd5aed-b12d-4d95-a31b-1f251aa98784 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:04.126210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-4e53c2fe-0750-45f7-b3c4-b3d13aa631c6 None None] MFA Rules not processed for user `b232b71c77a24be5bfcdb6aeb5807f90`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:04.130573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4e53c2fe-0750-45f7-b3c4-b3d13aa631c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:04.185475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c79e74-65c9-4eef-b9a6-67fbbcb0f7e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:04.186123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c79e74-65c9-4eef-b9a6-67fbbcb0f7e0 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.186408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c79e74-65c9-4eef-b9a6-67fbbcb0f7e0 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.186686 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c79e74-65c9-4eef-b9a6-67fbbcb0f7e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.188327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b7c79e74-65c9-4eef-b9a6-67fbbcb0f7e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:04.224904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:04.251134 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 704/1398] 10.222.0.26 () {58 vars in 1040 bytes} [Tue Oct 4 08:39:04 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:39:04.260514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53c2fe-0750-45f7-b3c4-b3d13aa631c6 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:04.270281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53c2fe-0750-45f7-b3c4-b3d13aa631c6 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:04.286227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c79e74-65c9-4eef-b9a6-67fbbcb0f7e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:04.287877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c79e74-65c9-4eef-b9a6-67fbbcb0f7e0 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:04.297598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:04.305999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 695/1399] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:04.385017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981cc36-6646-4469-9782-6cbfa2eb9b03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:04.385743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981cc36-6646-4469-9782-6cbfa2eb9b03 service swift] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:04.386170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981cc36-6646-4469-9782-6cbfa2eb9b03 service swift] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:04.386447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981cc36-6646-4469-9782-6cbfa2eb9b03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:04.388232 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7981cc36-6646-4469-9782-6cbfa2eb9b03 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:04.482317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981cc36-6646-4469-9782-6cbfa2eb9b03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:04.483999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981cc36-6646-4469-9782-6cbfa2eb9b03 service swift] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:04.493126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 705/1400] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:39:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 08:39:06.513239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebf7d5-1600-4708-8bd6-4bb59c74856e None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:06.513812 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebf7d5-1600-4708-8bd6-4bb59c74856e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:06.513911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebf7d5-1600-4708-8bd6-4bb59c74856e None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:06.515311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 696/1401] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:06.520616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd632400-bf76-4bc0-9ee1-e70c8447e1bb None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:06.521068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd632400-bf76-4bc0-9ee1-e70c8447e1bb None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:06.521301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd632400-bf76-4bc0-9ee1-e70c8447e1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:06.710763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-dd632400-bf76-4bc0-9ee1-e70c8447e1bb None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:06.712099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-dd632400-bf76-4bc0-9ee1-e70c8447e1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:06.720201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:06.818746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 706/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:07.113901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebf7d5-1600-4708-8bd6-4bb59c74856e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:07.116824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-deebf7d5-1600-4708-8bd6-4bb59c74856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:07.296297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebf7d5-1600-4708-8bd6-4bb59c74856e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:07.322333 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:07.461106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3eebe-264a-414d-8325-c1df1ecbc858 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:07.461622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3eebe-264a-414d-8325-c1df1ecbc858 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:07.461947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3eebe-264a-414d-8325-c1df1ecbc858 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:07.462225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3eebe-264a-414d-8325-c1df1ecbc858 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:07.463862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a0e3eebe-264a-414d-8325-c1df1ecbc858 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:07.558318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3eebe-264a-414d-8325-c1df1ecbc858 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:07.560152 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3eebe-264a-414d-8325-c1df1ecbc858 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:07.581519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 697/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:09.811052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ac4d2-ffb0-43c0-a8e2-3e386f43b8fc None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.811667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ac4d2-ffb0-43c0-a8e2-3e386f43b8fc None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.811667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ac4d2-ffb0-43c0-a8e2-3e386f43b8fc None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.813302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 707/1404] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:09.818600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743d58e-9881-4353-bb56-a2569b45210e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:09.819029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743d58e-9881-4353-bb56-a2569b45210e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:09.819180 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743d58e-9881-4353-bb56-a2569b45210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:09.843047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:09.902853 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-9743d58e-9881-4353-bb56-a2569b45210e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:09.904081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9743d58e-9881-4353-bb56-a2569b45210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:10.013343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 698/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:10.309330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ac4d2-ffb0-43c0-a8e2-3e386f43b8fc None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:10.311492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7f8ac4d2-ffb0-43c0-a8e2-3e386f43b8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:10.466984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:10.474539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ac4d2-ffb0-43c0-a8e2-3e386f43b8fc None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:10.592645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016f11c-5b9f-4d62-9e6f-829c90b0cdfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:10.593156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016f11c-5b9f-4d62-9e6f-829c90b0cdfd service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:10.593429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016f11c-5b9f-4d62-9e6f-829c90b0cdfd service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:10.593712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016f11c-5b9f-4d62-9e6f-829c90b0cdfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:10.595641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9016f11c-5b9f-4d62-9e6f-829c90b0cdfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:10.703375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016f11c-5b9f-4d62-9e6f-829c90b0cdfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:10.704994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016f11c-5b9f-4d62-9e6f-829c90b0cdfd service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:10.715609 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:10.724341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 708/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:12.963939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bded8-ab5d-428a-90ca-4dd652b2af11 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:12.964463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bded8-ab5d-428a-90ca-4dd652b2af11 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:12.964463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bded8-ab5d-428a-90ca-4dd652b2af11 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:12.965765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 699/1407] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:12.971382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf355e-4a56-45eb-ae12-9989234f553a None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:12.972162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf355e-4a56-45eb-ae12-9989234f553a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:12.972438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf355e-4a56-45eb-ae12-9989234f553a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:13.045256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-83cf355e-4a56-45eb-ae12-9989234f553a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:13.046878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-83cf355e-4a56-45eb-ae12-9989234f553a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:13.152614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 709/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:13.443314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bded8-ab5d-428a-90ca-4dd652b2af11 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:13.443314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3c3bded8-ab5d-428a-90ca-4dd652b2af11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:13.542494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:13.603975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bded8-ab5d-428a-90ca-4dd652b2af11 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:13.734516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb850f-9c77-49b9-846c-f2f83da6a524 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:13.735653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb850f-9c77-49b9-846c-f2f83da6a524 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:13.735931 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb850f-9c77-49b9-846c-f2f83da6a524 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:13.736200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb850f-9c77-49b9-846c-f2f83da6a524 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:13.739813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7deb850f-9c77-49b9-846c-f2f83da6a524 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:13.805286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:13.840177 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb850f-9c77-49b9-846c-f2f83da6a524 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:13.841817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb850f-9c77-49b9-846c-f2f83da6a524 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:13.865317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 700/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:23.151031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=67 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:23.862998 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=20 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:26.957215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de2fd0-2997-4ba0-891d-6b4c956d1d4c None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:26.957782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de2fd0-2997-4ba0-891d-6b4c956d1d4c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:26.957782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de2fd0-2997-4ba0-891d-6b4c956d1d4c None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:26.959085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 710/1410] 10.222.0.26 () {54 vars in 918 bytes} [Tue Oct 4 08:39:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:26.964454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c56cde-c7dd-4ce2-95f5-c10533998ffa None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:26.964899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c56cde-c7dd-4ce2-95f5-c10533998ffa None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:26.965003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c56cde-c7dd-4ce2-95f5-c10533998ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:27.040871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c4c56cde-c7dd-4ce2-95f5-c10533998ffa None None] MFA Rules not processed for user `abd158648914409f87ce0599843f0fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:27.042157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c4c56cde-c7dd-4ce2-95f5-c10533998ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:27.147374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 701/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Oct 4 08:39:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:27.161977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670da21-1e6d-4ca7-84c2-dac8709b158b None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:27.162258 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670da21-1e6d-4ca7-84c2-dac8709b158b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:27.162528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670da21-1e6d-4ca7-84c2-dac8709b158b None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:27.163864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 711/1412] 10.222.0.26 () {54 vars in 918 bytes} [Tue Oct 4 08:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:27.168546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbba8fe-b8dc-4cf1-bdb7-9410a541d297 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:27.168840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbba8fe-b8dc-4cf1-bdb7-9410a541d297 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:27.169113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbba8fe-b8dc-4cf1-bdb7-9410a541d297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:27.243005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:27.246546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-1cbba8fe-b8dc-4cf1-bdb7-9410a541d297 None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:27.248343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1cbba8fe-b8dc-4cf1-bdb7-9410a541d297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:27.363956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 702/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Oct 4 08:39:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:29.943309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc056eb-fbe1-4666-9cec-c6534d2c9f7c None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:29.943864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc056eb-fbe1-4666-9cec-c6534d2c9f7c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:29.943864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc056eb-fbe1-4666-9cec-c6534d2c9f7c None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:29.945084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:29.950329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132eeb93-be9b-4921-b432-3369d3af0616 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:29.950773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132eeb93-be9b-4921-b432-3369d3af0616 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:29.950878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132eeb93-be9b-4921-b432-3369d3af0616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.017223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-132eeb93-be9b-4921-b432-3369d3af0616 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:30.018442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-132eeb93-be9b-4921-b432-3369d3af0616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:30.072172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:30.114381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:30.405993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc056eb-fbe1-4666-9cec-c6534d2c9f7c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:30.407598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ecc056eb-fbe1-4666-9cec-c6534d2c9f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:30.551970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc056eb-fbe1-4666-9cec-c6534d2c9f7c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:30.622642 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:30.662274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0b615-faa1-45c1-b6f4-ecb86bc0bbc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:30.662871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0b615-faa1-45c1-b6f4-ecb86bc0bbc9 service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:30.663142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0b615-faa1-45c1-b6f4-ecb86bc0bbc9 service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:30.663412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0b615-faa1-45c1-b6f4-ecb86bc0bbc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:30.664997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-91f0b615-faa1-45c1-b6f4-ecb86bc0bbc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:30.756569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0b615-faa1-45c1-b6f4-ecb86bc0bbc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:30.758197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0b615-faa1-45c1-b6f4-ecb86bc0bbc9 service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:30.775889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 713/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:31.918287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8b008-a404-4aec-b382-6be135bd7ef4 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:31.918822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8b008-a404-4aec-b382-6be135bd7ef4 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:31.918822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8b008-a404-4aec-b382-6be135bd7ef4 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:31.920165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:31.926236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db40207-a1b5-4908-a49c-3ed0d7b5b264 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:31.926656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db40207-a1b5-4908-a49c-3ed0d7b5b264 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:31.926771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db40207-a1b5-4908-a49c-3ed0d7b5b264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:32.001244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-4db40207-a1b5-4908-a49c-3ed0d7b5b264 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:32.002462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4db40207-a1b5-4908-a49c-3ed0d7b5b264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:32.028655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:32.097649 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:32.247123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8b008-a404-4aec-b382-6be135bd7ef4 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:32.248575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-75b8b008-a404-4aec-b382-6be135bd7ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:32.393318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8b008-a404-4aec-b382-6be135bd7ef4 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:32.433898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:32.505427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8032378-ab6a-49b6-b2d0-d6bbd52940c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:32.505995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8032378-ab6a-49b6-b2d0-d6bbd52940c9 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:32.506275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8032378-ab6a-49b6-b2d0-d6bbd52940c9 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:32.506546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8032378-ab6a-49b6-b2d0-d6bbd52940c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:32.508159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c8032378-ab6a-49b6-b2d0-d6bbd52940c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:32.596684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8032378-ab6a-49b6-b2d0-d6bbd52940c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:32.598342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8032378-ab6a-49b6-b2d0-d6bbd52940c9 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:32.615913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 705/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:33.751836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd5c3-5ab2-4591-a0bc-ff1a0d60ae55 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:33.752257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd5c3-5ab2-4591-a0bc-ff1a0d60ae55 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:33.752389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcd5c3-5ab2-4591-a0bc-ff1a0d60ae55 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:33.753760 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:33.760024 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b1062-2dc3-47e7-87c9-2fa8d07cb7de None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:33.760448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b1062-2dc3-47e7-87c9-2fa8d07cb7de None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:33.760597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b1062-2dc3-47e7-87c9-2fa8d07cb7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:33.815508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:33.837973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-011b1062-2dc3-47e7-87c9-2fa8d07cb7de None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:33.839169 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-011b1062-2dc3-47e7-87c9-2fa8d07cb7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:33.933756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:34.086491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd5c3-5ab2-4591-a0bc-ff1a0d60ae55 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:34.088082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-74bcd5c3-5ab2-4591-a0bc-ff1a0d60ae55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:34.233015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcd5c3-5ab2-4591-a0bc-ff1a0d60ae55 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:34.244841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:34.346170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d9da3-8c2e-46fe-ae01-518b5840874d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:34.346763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d9da3-8c2e-46fe-ae01-518b5840874d service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:34.347035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d9da3-8c2e-46fe-ae01-518b5840874d service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:34.347298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d9da3-8c2e-46fe-ae01-518b5840874d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:34.348873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-128d9da3-8c2e-46fe-ae01-518b5840874d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:34.434758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d9da3-8c2e-46fe-ae01-518b5840874d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:34.436343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d9da3-8c2e-46fe-ae01-518b5840874d service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:34.454922 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 716/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:35.550394 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d573ee-369e-4c36-a5fd-2f3332908dd6 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:35.550940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d573ee-369e-4c36-a5fd-2f3332908dd6 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:35.550940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d573ee-369e-4c36-a5fd-2f3332908dd6 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:35.552269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:35.558289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eef065-37da-480b-b5d6-a1833266ab15 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:35.558684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eef065-37da-480b-b5d6-a1833266ab15 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:35.558872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eef065-37da-480b-b5d6-a1833266ab15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:35.578685 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:35.634792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-06eef065-37da-480b-b5d6-a1833266ab15 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:35.636000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-06eef065-37da-480b-b5d6-a1833266ab15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:35.730112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:35.880728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d573ee-369e-4c36-a5fd-2f3332908dd6 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:35.882215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-24d573ee-369e-4c36-a5fd-2f3332908dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:35.989646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:36.023927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d573ee-369e-4c36-a5fd-2f3332908dd6 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:36.137639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ae47a-4352-45da-ba8a-11fd8dd57fc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:36.138220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ae47a-4352-45da-ba8a-11fd8dd57fc7 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:36.138503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ae47a-4352-45da-ba8a-11fd8dd57fc7 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:36.138777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ae47a-4352-45da-ba8a-11fd8dd57fc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:36.140352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6a5ae47a-4352-45da-ba8a-11fd8dd57fc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:36.221390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:36.235305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ae47a-4352-45da-ba8a-11fd8dd57fc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:36.236919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ae47a-4352-45da-ba8a-11fd8dd57fc7 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:36.255115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 708/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:37.404342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452db502-f176-4ecb-9d50-c9dfa9432b04 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:37.404875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452db502-f176-4ecb-9d50-c9dfa9432b04 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:37.404875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452db502-f176-4ecb-9d50-c9dfa9432b04 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:37.406700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:37.412831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498757f-d47d-4c67-ada3-e345a5d4b30b None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:37.413270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498757f-d47d-4c67-ada3-e345a5d4b30b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:37.413383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498757f-d47d-4c67-ada3-e345a5d4b30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:37.489287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-1498757f-d47d-4c67-ada3-e345a5d4b30b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:37.490517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1498757f-d47d-4c67-ada3-e345a5d4b30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:37.585099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:37.735980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452db502-f176-4ecb-9d50-c9dfa9432b04 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:37.737539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-452db502-f176-4ecb-9d50-c9dfa9432b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:37.820116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:37.881390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452db502-f176-4ecb-9d50-c9dfa9432b04 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:37.992996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bbc3a-8a4f-46c8-92c0-944fd76f6cdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:37.993525 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bbc3a-8a4f-46c8-92c0-944fd76f6cdf service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:37.993843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bbc3a-8a4f-46c8-92c0-944fd76f6cdf service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:37.994110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bbc3a-8a4f-46c8-92c0-944fd76f6cdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:37.995701 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a02bbc3a-8a4f-46c8-92c0-944fd76f6cdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:38.042911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:38.083653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bbc3a-8a4f-46c8-92c0-944fd76f6cdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:38.085229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bbc3a-8a4f-46c8-92c0-944fd76f6cdf service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:38.102902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 719/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:39.153152 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a9d32-d6ea-47a4-97bb-b16fccfe6f7a None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:39.153798 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a9d32-d6ea-47a4-97bb-b16fccfe6f7a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:39.153798 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a9d32-d6ea-47a4-97bb-b16fccfe6f7a None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:39.155415 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:39.163064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a2944-b914-4269-acbe-cadff961984d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:39.163582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a2944-b914-4269-acbe-cadff961984d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:39.163677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a2944-b914-4269-acbe-cadff961984d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:39.239544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-794a2944-b914-4269-acbe-cadff961984d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:39.240769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-794a2944-b914-4269-acbe-cadff961984d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:39.341699 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:39.490398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a9d32-d6ea-47a4-97bb-b16fccfe6f7a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:39.491898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-bb2a9d32-d6ea-47a4-97bb-b16fccfe6f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:39.540368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:39.637104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a9d32-d6ea-47a4-97bb-b16fccfe6f7a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:39.748768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8c027-d363-49d0-b06a-fcb647dce5cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:39.749411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8c027-d363-49d0-b06a-fcb647dce5cd service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:39.749716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8c027-d363-49d0-b06a-fcb647dce5cd service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:39.750490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8c027-d363-49d0-b06a-fcb647dce5cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:39.752180 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-02e8c027-d363-49d0-b06a-fcb647dce5cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:39.759172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:39.840639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8c027-d363-49d0-b06a-fcb647dce5cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:39.842411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8c027-d363-49d0-b06a-fcb647dce5cd service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:39.866146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 711/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:40.915708 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318a9f3-e3e9-4791-a89a-e0fa6863f03a None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:40.916235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318a9f3-e3e9-4791-a89a-e0fa6863f03a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:40.916235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318a9f3-e3e9-4791-a89a-e0fa6863f03a None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:40.917520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:40.924307 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c8793-02c2-467d-b825-fc8360e565ee None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:40.924767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c8793-02c2-467d-b825-fc8360e565ee None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:40.924874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c8793-02c2-467d-b825-fc8360e565ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:40.998688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f64c8793-02c2-467d-b825-fc8360e565ee None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:40.999873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f64c8793-02c2-467d-b825-fc8360e565ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:41.068965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:41.098352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:41.247762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318a9f3-e3e9-4791-a89a-e0fa6863f03a None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:41.249250 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e318a9f3-e3e9-4791-a89a-e0fa6863f03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:41.266631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:41.392960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318a9f3-e3e9-4791-a89a-e0fa6863f03a None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:41.478940 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:41.503818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076e61e-ae8c-4721-ab77-320e0301c39a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:41.504382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076e61e-ae8c-4721-ab77-320e0301c39a service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:41.504655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076e61e-ae8c-4721-ab77-320e0301c39a service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:41.504920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076e61e-ae8c-4721-ab77-320e0301c39a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:41.506530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-5076e61e-ae8c-4721-ab77-320e0301c39a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:41.595289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076e61e-ae8c-4721-ab77-320e0301c39a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:41.597116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076e61e-ae8c-4721-ab77-320e0301c39a service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:41.617741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 722/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:42.664633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23649e3e-5455-4503-9447-c09c0573168b None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:42.665131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23649e3e-5455-4503-9447-c09c0573168b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:42.665235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23649e3e-5455-4503-9447-c09c0573168b None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:42.666566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:42.672875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59c40e-f640-4a12-a1ac-6ba7680a1d2e None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:42.673315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59c40e-f640-4a12-a1ac-6ba7680a1d2e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:42.673446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59c40e-f640-4a12-a1ac-6ba7680a1d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:42.746895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-cc59c40e-f640-4a12-a1ac-6ba7680a1d2e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:42.748086 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-cc59c40e-f640-4a12-a1ac-6ba7680a1d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:42.788417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:42.842763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:42.993336 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23649e3e-5455-4503-9447-c09c0573168b None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:42.994839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-23649e3e-5455-4503-9447-c09c0573168b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:43.136328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23649e3e-5455-4503-9447-c09c0573168b None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:43.189052 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:43.248208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b5b2d-602f-4aab-a924-657e0d90f2a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:43.248761 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b5b2d-602f-4aab-a924-657e0d90f2a6 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:43.249040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b5b2d-602f-4aab-a924-657e0d90f2a6 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:43.249308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b5b2d-602f-4aab-a924-657e0d90f2a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:43.251638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1c0b5b2d-602f-4aab-a924-657e0d90f2a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:43.340534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b5b2d-602f-4aab-a924-657e0d90f2a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:43.342168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b5b2d-602f-4aab-a924-657e0d90f2a6 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:43.361007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 714/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:44.454206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaada6f-61dd-4e76-940f-4aeb9304412c None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.454810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaada6f-61dd-4e76-940f-4aeb9304412c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.454810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaada6f-61dd-4e76-940f-4aeb9304412c None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.456119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:39:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:44.462544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad31a8-c65e-4514-8118-7c60e1dd95ee None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:44.462979 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad31a8-c65e-4514-8118-7c60e1dd95ee None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:44.463082 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad31a8-c65e-4514-8118-7c60e1dd95ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:44.541944 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-dcad31a8-c65e-4514-8118-7c60e1dd95ee None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:44.543160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-dcad31a8-c65e-4514-8118-7c60e1dd95ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:44.545196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:44.638610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:44.787832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaada6f-61dd-4e76-940f-4aeb9304412c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:44.789369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2eaada6f-61dd-4e76-940f-4aeb9304412c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:44.956921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaada6f-61dd-4e76-940f-4aeb9304412c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:44.986462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:45.072541 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83477a55-391e-41bd-a099-8f4ee394e2f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:45.073052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83477a55-391e-41bd-a099-8f4ee394e2f8 service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:45.073330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83477a55-391e-41bd-a099-8f4ee394e2f8 service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:45.073598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83477a55-391e-41bd-a099-8f4ee394e2f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:45.075313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-83477a55-391e-41bd-a099-8f4ee394e2f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:45.162636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83477a55-391e-41bd-a099-8f4ee394e2f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:45.164223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83477a55-391e-41bd-a099-8f4ee394e2f8 service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:45.182150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 725/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:47.207774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81850af-cd1c-49d9-ae01-d11b6b5d8979 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.208398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81850af-cd1c-49d9-ae01-d11b6b5d8979 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.208398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81850af-cd1c-49d9-ae01-d11b6b5d8979 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.209755 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:47.214932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d7343-0213-47ee-90de-fd890dadac9d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.215357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d7343-0213-47ee-90de-fd890dadac9d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.215479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d7343-0213-47ee-90de-fd890dadac9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.255938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:47.282897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-e90d7343-0213-47ee-90de-fd890dadac9d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:47.284379 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e90d7343-0213-47ee-90de-fd890dadac9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:47.380969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:47.664104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81850af-cd1c-49d9-ae01-d11b6b5d8979 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:47.665658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e81850af-cd1c-49d9-ae01-d11b6b5d8979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:47.768353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81850af-cd1c-49d9-ae01-d11b6b5d8979 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:47.800418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:47.841568 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b6945-c068-4ca7-b562-ed2443d6b36b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:47.842196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b6945-c068-4ca7-b562-ed2443d6b36b service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:47.842476 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b6945-c068-4ca7-b562-ed2443d6b36b service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:47.842747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b6945-c068-4ca7-b562-ed2443d6b36b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:47.844326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6f9b6945-c068-4ca7-b562-ed2443d6b36b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:47.950520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b6945-c068-4ca7-b562-ed2443d6b36b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:47.951904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b6945-c068-4ca7-b562-ed2443d6b36b service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:47.969760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 717/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:50.314743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00ce66-2e27-499c-a5a0-4ba8b7a1b95a None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:50.315327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00ce66-2e27-499c-a5a0-4ba8b7a1b95a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:50.315327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00ce66-2e27-499c-a5a0-4ba8b7a1b95a None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:50.316733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:50.321955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a337-27b0-447f-a732-21b1cbbef656 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:50.322364 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a337-27b0-447f-a732-21b1cbbef656 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:50.322531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a337-27b0-447f-a732-21b1cbbef656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:50.389417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-e370a337-27b0-447f-a732-21b1cbbef656 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:50.390786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e370a337-27b0-447f-a732-21b1cbbef656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:50.403939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:50.605902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:39:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:50.890389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00ce66-2e27-499c-a5a0-4ba8b7a1b95a None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:50.892567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3b00ce66-2e27-499c-a5a0-4ba8b7a1b95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:50.994710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00ce66-2e27-499c-a5a0-4ba8b7a1b95a None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:51.007887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:51.067120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657276a-9c0f-49fa-977b-20ce4d226bbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:51.067723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657276a-9c0f-49fa-977b-20ce4d226bbe service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:51.068001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657276a-9c0f-49fa-977b-20ce4d226bbe service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:51.068272 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657276a-9c0f-49fa-977b-20ce4d226bbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:51.069892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2657276a-9c0f-49fa-977b-20ce4d226bbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:51.176595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657276a-9c0f-49fa-977b-20ce4d226bbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:51.178274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657276a-9c0f-49fa-977b-20ce4d226bbe service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:51.195969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 728/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:53.736053 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98464c3-c525-467e-8924-53478c9ab457 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:53.736608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98464c3-c525-467e-8924-53478c9ab457 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:53.736608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98464c3-c525-467e-8924-53478c9ab457 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:53.737972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:53.744227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e2263-5dd9-4489-8738-ae35c8feae42 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:53.744689 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e2263-5dd9-4489-8738-ae35c8feae42 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:53.745140 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e2263-5dd9-4489-8738-ae35c8feae42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:53.811054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:53.833412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-3b1e2263-5dd9-4489-8738-ae35c8feae42 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:53.834723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3b1e2263-5dd9-4489-8738-ae35c8feae42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:53.929745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:54.215336 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98464c3-c525-467e-8924-53478c9ab457 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:54.216864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a98464c3-c525-467e-8924-53478c9ab457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:54.318859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98464c3-c525-467e-8924-53478c9ab457 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:54.358841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:54.392488 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9ee9a-ef75-4e32-bbaa-9e4491dbb525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:54.393090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9ee9a-ef75-4e32-bbaa-9e4491dbb525 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:54.393369 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9ee9a-ef75-4e32-bbaa-9e4491dbb525 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:54.393642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9ee9a-ef75-4e32-bbaa-9e4491dbb525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:54.395266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-31a9ee9a-ef75-4e32-bbaa-9e4491dbb525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:54.502551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9ee9a-ef75-4e32-bbaa-9e4491dbb525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:54.504004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9ee9a-ef75-4e32-bbaa-9e4491dbb525 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:54.522606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 720/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:56.776971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea3f3d-7694-4358-8245-49362fde7bfc None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:56.777507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea3f3d-7694-4358-8245-49362fde7bfc None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:56.777624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea3f3d-7694-4358-8245-49362fde7bfc None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:56.779008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:56.784980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76e80d-0825-4e99-82be-b0b5638dd812 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:56.785440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76e80d-0825-4e99-82be-b0b5638dd812 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:56.785540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76e80d-0825-4e99-82be-b0b5638dd812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:56.851375 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-6f76e80d-0825-4e99-82be-b0b5638dd812 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:56.852612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6f76e80d-0825-4e99-82be-b0b5638dd812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:56.873306 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:56.948828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:39:57.239361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea3f3d-7694-4358-8245-49362fde7bfc None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:39:57.240910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-cfea3f3d-7694-4358-8245-49362fde7bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:57.343583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea3f3d-7694-4358-8245-49362fde7bfc None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:39:57.365236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:57.417002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363b019-370f-4c1c-aeb0-59c700e36d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:39:57.417600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363b019-370f-4c1c-aeb0-59c700e36d56 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:57.417907 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363b019-370f-4c1c-aeb0-59c700e36d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:57.418180 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363b019-370f-4c1c-aeb0-59c700e36d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:57.419778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d363b019-370f-4c1c-aeb0-59c700e36d56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:57.525387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363b019-370f-4c1c-aeb0-59c700e36d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:39:57.526860 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363b019-370f-4c1c-aeb0-59c700e36d56 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:39:57.544641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 731/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:39:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:39:59.739851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76a97b-03b5-478c-b50e-6d6655c3b415 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:59.740399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76a97b-03b5-478c-b50e-6d6655c3b415 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:59.740399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76a97b-03b5-478c-b50e-6d6655c3b415 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:59.741746 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:39:59.747071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240495f-84b8-4297-b02b-efb84ba37f5b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:39:59.747494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240495f-84b8-4297-b02b-efb84ba37f5b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:39:59.747644 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240495f-84b8-4297-b02b-efb84ba37f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:39:59.810634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:39:59.814209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-7240495f-84b8-4297-b02b-efb84ba37f5b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:39:59.815779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7240495f-84b8-4297-b02b-efb84ba37f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:39:59.910768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:00.195508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76a97b-03b5-478c-b50e-6d6655c3b415 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:00.197059 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1c76a97b-03b5-478c-b50e-6d6655c3b415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:00.300154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76a97b-03b5-478c-b50e-6d6655c3b415 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:00.349339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:00.372971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f4649-3746-43e7-9ee5-a5aec051b870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:00.373589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f4649-3746-43e7-9ee5-a5aec051b870 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:00.373896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f4649-3746-43e7-9ee5-a5aec051b870 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:00.374167 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f4649-3746-43e7-9ee5-a5aec051b870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:00.375822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6a6f4649-3746-43e7-9ee5-a5aec051b870 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:00.481571 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f4649-3746-43e7-9ee5-a5aec051b870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:00.482952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f4649-3746-43e7-9ee5-a5aec051b870 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:00.500861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 723/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:40:02.702228 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a71cc-9fc5-4894-b80f-357b74f41c92 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:02.702783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a71cc-9fc5-4894-b80f-357b74f41c92 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:02.702783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a71cc-9fc5-4894-b80f-357b74f41c92 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:02.704189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:02.709456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33777860-d043-4229-95a2-dfe05feae856 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:02.709886 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33777860-d043-4229-95a2-dfe05feae856 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:02.710040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33777860-d043-4229-95a2-dfe05feae856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:02.775353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-33777860-d043-4229-95a2-dfe05feae856 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:02.776541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-33777860-d043-4229-95a2-dfe05feae856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:02.805321 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:02.872132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:40:03.158022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a71cc-9fc5-4894-b80f-357b74f41c92 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:03.159634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a60a71cc-9fc5-4894-b80f-357b74f41c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:03.265571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a71cc-9fc5-4894-b80f-357b74f41c92 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:03.295147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:03.340903 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b9f1d-16c4-4918-9e56-fef15be45982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:03.341498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b9f1d-16c4-4918-9e56-fef15be45982 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:03.341802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b9f1d-16c4-4918-9e56-fef15be45982 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:03.342074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b9f1d-16c4-4918-9e56-fef15be45982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:03.343655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-473b9f1d-16c4-4918-9e56-fef15be45982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:03.451583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b9f1d-16c4-4918-9e56-fef15be45982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:03.452946 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b9f1d-16c4-4918-9e56-fef15be45982 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:03.470695 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 734/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:40:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:40:05.629635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12392695-ec47-4856-b38c-bfbe36734dfb None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:05.630225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12392695-ec47-4856-b38c-bfbe36734dfb None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:05.630225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12392695-ec47-4856-b38c-bfbe36734dfb None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:05.631529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:05.636750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc02fe-bf00-48a1-b9b0-5ac407f2e951 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:05.637183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc02fe-bf00-48a1-b9b0-5ac407f2e951 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:05.637311 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc02fe-bf00-48a1-b9b0-5ac407f2e951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:05.703011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-dcfc02fe-bf00-48a1-b9b0-5ac407f2e951 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:05.704221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-dcfc02fe-bf00-48a1-b9b0-5ac407f2e951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:05.712319 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:05.800828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:06.082012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12392695-ec47-4856-b38c-bfbe36734dfb None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:06.083536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-12392695-ec47-4856-b38c-bfbe36734dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:06.186624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12392695-ec47-4856-b38c-bfbe36734dfb None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:06.241897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:06.259703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c32ed-14a3-4352-b5bb-fd9d00d4ec90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:06.260298 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c32ed-14a3-4352-b5bb-fd9d00d4ec90 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:06.260576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c32ed-14a3-4352-b5bb-fd9d00d4ec90 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:06.260844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c32ed-14a3-4352-b5bb-fd9d00d4ec90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:06.262442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0b2c32ed-14a3-4352-b5bb-fd9d00d4ec90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:06.370188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c32ed-14a3-4352-b5bb-fd9d00d4ec90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:06.371593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c32ed-14a3-4352-b5bb-fd9d00d4ec90 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:06.389746 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 726/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:40:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:40:08.580736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca7501-3ff0-40ea-91d3-ed0201196f74 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:08.581298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca7501-3ff0-40ea-91d3-ed0201196f74 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:08.581298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca7501-3ff0-40ea-91d3-ed0201196f74 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:08.582686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:08.587905 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c997879-6948-44b5-9e0c-7a104c2d3489 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:08.588297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c997879-6948-44b5-9e0c-7a104c2d3489 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:08.588464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c997879-6948-44b5-9e0c-7a104c2d3489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:08.654560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-9c997879-6948-44b5-9e0c-7a104c2d3489 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:08.655741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9c997879-6948-44b5-9e0c-7a104c2d3489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:08.693892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:08.751762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:09.042806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca7501-3ff0-40ea-91d3-ed0201196f74 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:09.044361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-aeca7501-3ff0-40ea-91d3-ed0201196f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:09.147004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca7501-3ff0-40ea-91d3-ed0201196f74 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:09.187335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:09.225362 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7813072-89ad-497a-aca3-978bb3d280ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:09.225997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7813072-89ad-497a-aca3-978bb3d280ca service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:09.226278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7813072-89ad-497a-aca3-978bb3d280ca service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:09.226550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7813072-89ad-497a-aca3-978bb3d280ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:09.228152 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d7813072-89ad-497a-aca3-978bb3d280ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:09.334904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7813072-89ad-497a-aca3-978bb3d280ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:09.336258 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7813072-89ad-497a-aca3-978bb3d280ca service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:09.355177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 737/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:40:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:40:11.512689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277fe20-fd48-4945-92d8-16ce2cc06308 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:11.513230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277fe20-fd48-4945-92d8-16ce2cc06308 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:11.513230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8277fe20-fd48-4945-92d8-16ce2cc06308 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:11.514554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:11.520352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d2a0-c410-4ef9-929f-5e4b612cf30d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:11.520784 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d2a0-c410-4ef9-929f-5e4b612cf30d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:11.520915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771d2a0-c410-4ef9-929f-5e4b612cf30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:11.586714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-5771d2a0-c410-4ef9-929f-5e4b612cf30d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:11.587924 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-5771d2a0-c410-4ef9-929f-5e4b612cf30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:11.606805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:11.686182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:11.967774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277fe20-fd48-4945-92d8-16ce2cc06308 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:11.969334 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8277fe20-fd48-4945-92d8-16ce2cc06308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:12.071390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8277fe20-fd48-4945-92d8-16ce2cc06308 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:12.136943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:12.144541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db64f3f-f783-4dd3-8c1c-c4d0e876821b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:12.145145 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db64f3f-f783-4dd3-8c1c-c4d0e876821b service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:12.145421 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db64f3f-f783-4dd3-8c1c-c4d0e876821b service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:12.145709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db64f3f-f783-4dd3-8c1c-c4d0e876821b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:12.147339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8db64f3f-f783-4dd3-8c1c-c4d0e876821b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:12.253416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db64f3f-f783-4dd3-8c1c-c4d0e876821b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:12.254827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db64f3f-f783-4dd3-8c1c-c4d0e876821b service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:12.272299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 729/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:40:14.431860 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179673f-598a-46c4-a3c7-7d54bda45ba1 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:14.432439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179673f-598a-46c4-a3c7-7d54bda45ba1 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:14.432439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179673f-598a-46c4-a3c7-7d54bda45ba1 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:14.433748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:14.439236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934e503-e012-43a8-a4ca-064d983ca07d None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:14.439697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934e503-e012-43a8-a4ca-064d983ca07d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:14.439808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934e503-e012-43a8-a4ca-064d983ca07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:14.504939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-b934e503-e012-43a8-a4ca-064d983ca07d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:14.506181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b934e503-e012-43a8-a4ca-064d983ca07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:14.554301 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:14.602899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:14.883424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179673f-598a-46c4-a3c7-7d54bda45ba1 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:14.884968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a179673f-598a-46c4-a3c7-7d54bda45ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:14.986745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179673f-598a-46c4-a3c7-7d54bda45ba1 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:15.035053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:15.061261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5283d4-843e-4087-a79e-97d5b22018e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:15.061895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5283d4-843e-4087-a79e-97d5b22018e0 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:15.062677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5283d4-843e-4087-a79e-97d5b22018e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:15.062948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5283d4-843e-4087-a79e-97d5b22018e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:15.064545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1e5283d4-843e-4087-a79e-97d5b22018e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:15.174321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5283d4-843e-4087-a79e-97d5b22018e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:15.175678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5283d4-843e-4087-a79e-97d5b22018e0 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:15.193797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 740/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:40:24.601335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=22 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:25.191598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=66 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:34.984542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ed03c-34f9-412e-b627-af353100998a None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:34.985066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ed03c-34f9-412e-b627-af353100998a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:34.985066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ed03c-34f9-412e-b627-af353100998a None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:34.986452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 731/1471] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:34.991767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc547b-ef62-4bda-84ff-a52b2b36b6bc None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:34.992197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc547b-ef62-4bda-84ff-a52b2b36b6bc None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:34.992310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc547b-ef62-4bda-84ff-a52b2b36b6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:35.057239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-06cc547b-ef62-4bda-84ff-a52b2b36b6bc None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:35.058477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-06cc547b-ef62-4bda-84ff-a52b2b36b6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:35.153678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 741/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:35.448777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b56843-9817-4de5-9aa3-e22b21ed80e3 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:35.449063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b56843-9817-4de5-9aa3-e22b21ed80e3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:35.449331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b56843-9817-4de5-9aa3-e22b21ed80e3 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:35.450689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 732/1473] 10.222.0.26 () {54 vars in 962 bytes} [Tue Oct 4 08:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:35.456063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa704da6-a273-4c9a-b4e7-5061caefd5ed None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:35.456351 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa704da6-a273-4c9a-b4e7-5061caefd5ed None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:35.456621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa704da6-a273-4c9a-b4e7-5061caefd5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:35.531641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-aa704da6-a273-4c9a-b4e7-5061caefd5ed None None] MFA Rules not processed for user `913a866ddc074b7f9aa142c70da98a69`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:35.532836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-aa704da6-a273-4c9a-b4e7-5061caefd5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:35.552622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:35.607272 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 742/1474] 10.222.0.26 () {58 vars in 1059 bytes} [Tue Oct 4 08:40:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:35.614936 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b56843-9817-4de5-9aa3-e22b21ed80e3 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:35.616420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a4b56843-9817-4de5-9aa3-e22b21ed80e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:35.716447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b56843-9817-4de5-9aa3-e22b21ed80e3 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:35.789786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c0584-38d7-4f21-8443-2573632753fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '913a866ddc074b7f9aa142c70da98a69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:35.790388 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c0584-38d7-4f21-8443-2573632753fe service glance] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:35.790665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c0584-38d7-4f21-8443-2573632753fe service glance] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:35.790934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c0584-38d7-4f21-8443-2573632753fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:35.792518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7d1c0584-38d7-4f21-8443-2573632753fe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:35.850254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:35.904450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c0584-38d7-4f21-8443-2573632753fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:35.905933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c0584-38d7-4f21-8443-2573632753fe service glance] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:35.923838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 733/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:40:38.160658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40eb30-ed86-47eb-835b-ed823b2f14a9 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:38.161228 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40eb30-ed86-47eb-835b-ed823b2f14a9 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:38.161228 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40eb30-ed86-47eb-835b-ed823b2f14a9 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:38.162724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 743/1476] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:38.167984 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392eff0-70ff-4dd5-9d17-adf895604386 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:38.168416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392eff0-70ff-4dd5-9d17-adf895604386 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:38.168514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392eff0-70ff-4dd5-9d17-adf895604386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:38.233633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-b392eff0-70ff-4dd5-9d17-adf895604386 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:38.235031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b392eff0-70ff-4dd5-9d17-adf895604386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:38.330506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 734/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:38.611539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40eb30-ed86-47eb-835b-ed823b2f14a9 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:38.613135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6d40eb30-ed86-47eb-835b-ed823b2f14a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:38.741904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40eb30-ed86-47eb-835b-ed823b2f14a9 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:38.791418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:38.835158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6e650-a87c-46ee-aeec-821227540c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:38.835777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6e650-a87c-46ee-aeec-821227540c82 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:38.836052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6e650-a87c-46ee-aeec-821227540c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:38.836319 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6e650-a87c-46ee-aeec-821227540c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:38.836948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6e650-a87c-46ee-aeec-821227540c82 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:38.838706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: WARNING py.warnings [None req-74b6e650-a87c-46ee-aeec-821227540c82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:40:38.838706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: warnings.warn(msg) Oct 04 08:40:38.838706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: Oct 04 08:40:38.839843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6e650-a87c-46ee-aeec-821227540c82 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:38.858637 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 744/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Tue Oct 4 08:40:38 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:40:41.026283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ced8e9-4595-4f5e-9250-adf077725325 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:41.026894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ced8e9-4595-4f5e-9250-adf077725325 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:41.026894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ced8e9-4595-4f5e-9250-adf077725325 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:41.028173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 735/1479] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:41.033896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20874d67-5982-4840-b466-bcd24697ed28 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:41.034306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20874d67-5982-4840-b466-bcd24697ed28 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:41.034441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20874d67-5982-4840-b466-bcd24697ed28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:41.101572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-20874d67-5982-4840-b466-bcd24697ed28 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:41.102833 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-20874d67-5982-4840-b466-bcd24697ed28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:41.196751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 745/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:41.490256 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ced8e9-4595-4f5e-9250-adf077725325 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:41.491827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-25ced8e9-4595-4f5e-9250-adf077725325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:41.502830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:41.636207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ced8e9-4595-4f5e-9250-adf077725325 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:41.714044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:41.746483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba332d4b-f3ee-4483-9aef-529ef55b8fec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:41.747014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba332d4b-f3ee-4483-9aef-529ef55b8fec service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:41.747290 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba332d4b-f3ee-4483-9aef-529ef55b8fec service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:41.747555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba332d4b-f3ee-4483-9aef-529ef55b8fec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:41.749133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ba332d4b-f3ee-4483-9aef-529ef55b8fec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:41.836203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba332d4b-f3ee-4483-9aef-529ef55b8fec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:41.837809 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba332d4b-f3ee-4483-9aef-529ef55b8fec service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:41.855887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 736/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:40:44.150463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870a4e9-9a2d-42ec-9d51-833fa2736139 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:44.151040 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870a4e9-9a2d-42ec-9d51-833fa2736139 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:44.151145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870a4e9-9a2d-42ec-9d51-833fa2736139 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:44.152499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 746/1482] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:44.157814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5061b79-f182-4f0b-8654-772790b497a1 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:44.158232 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5061b79-f182-4f0b-8654-772790b497a1 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:44.158370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5061b79-f182-4f0b-8654-772790b497a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:44.223956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a5061b79-f182-4f0b-8654-772790b497a1 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:44.225166 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a5061b79-f182-4f0b-8654-772790b497a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:44.271278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:44.321551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 737/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:44.602876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870a4e9-9a2d-42ec-9d51-833fa2736139 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:44.604473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e870a4e9-9a2d-42ec-9d51-833fa2736139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:44.629540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:44.707986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870a4e9-9a2d-42ec-9d51-833fa2736139 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:44.781539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d81495-40d8-4060-8781-94a344deea6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:44.782163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d81495-40d8-4060-8781-94a344deea6c service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:44.782461 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d81495-40d8-4060-8781-94a344deea6c service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:44.782727 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d81495-40d8-4060-8781-94a344deea6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:44.784305 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-50d81495-40d8-4060-8781-94a344deea6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:44.856785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:44.890809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d81495-40d8-4060-8781-94a344deea6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:44.892245 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d81495-40d8-4060-8781-94a344deea6c service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:44.910423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 747/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:40:47.084369 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da36492-46bb-4587-be7d-5f214307db99 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:47.084920 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da36492-46bb-4587-be7d-5f214307db99 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:47.084920 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da36492-46bb-4587-be7d-5f214307db99 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:47.086286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 738/1485] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:47.091603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98a5b0-4fb8-4871-b43e-3283776a3f60 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:47.091992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98a5b0-4fb8-4871-b43e-3283776a3f60 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:47.092145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98a5b0-4fb8-4871-b43e-3283776a3f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:47.157040 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-3d98a5b0-4fb8-4871-b43e-3283776a3f60 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:47.158263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3d98a5b0-4fb8-4871-b43e-3283776a3f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:47.253054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 748/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:47.543733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da36492-46bb-4587-be7d-5f214307db99 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:47.545260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3da36492-46bb-4587-be7d-5f214307db99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:47.645266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da36492-46bb-4587-be7d-5f214307db99 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:47.719487 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:47.721631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df127dc2-cd72-4332-9ca6-a6b61ff54215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:47.722346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df127dc2-cd72-4332-9ca6-a6b61ff54215 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:47.722828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df127dc2-cd72-4332-9ca6-a6b61ff54215 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:47.723260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df127dc2-cd72-4332-9ca6-a6b61ff54215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:47.725135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-df127dc2-cd72-4332-9ca6-a6b61ff54215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:47.844559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df127dc2-cd72-4332-9ca6-a6b61ff54215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:47.845941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df127dc2-cd72-4332-9ca6-a6b61ff54215 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:47.863777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 739/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:40:50.206808 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efc648-1cb9-4682-8428-ccdb7ad8820e None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:50.207369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efc648-1cb9-4682-8428-ccdb7ad8820e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:50.207369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efc648-1cb9-4682-8428-ccdb7ad8820e None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:50.208710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 749/1488] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:50.214085 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e9450-03a1-48d2-971e-b7a26e16f8d6 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:50.214478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e9450-03a1-48d2-971e-b7a26e16f8d6 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:50.214666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e9450-03a1-48d2-971e-b7a26e16f8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:50.279716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-8b6e9450-03a1-48d2-971e-b7a26e16f8d6 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:50.281441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8b6e9450-03a1-48d2-971e-b7a26e16f8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:50.335253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:50.379746 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 740/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:50.671641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efc648-1cb9-4682-8428-ccdb7ad8820e None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:50.673197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b3efc648-1cb9-4682-8428-ccdb7ad8820e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:50.708842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:50.775277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efc648-1cb9-4682-8428-ccdb7ad8820e None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:50.848751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5db60-db34-4577-a306-cf43572c7e9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:50.849354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5db60-db34-4577-a306-cf43572c7e9a service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:50.849632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5db60-db34-4577-a306-cf43572c7e9a service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:50.849929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5db60-db34-4577-a306-cf43572c7e9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:50.851512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fdf5db60-db34-4577-a306-cf43572c7e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:50.929460 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:50.958775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5db60-db34-4577-a306-cf43572c7e9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:50.960211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5db60-db34-4577-a306-cf43572c7e9a service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:50.978926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 750/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:40:53.310099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a0134-8ac8-4dd7-97ae-499b35f89740 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:53.310668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a0134-8ac8-4dd7-97ae-499b35f89740 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:53.310668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a0134-8ac8-4dd7-97ae-499b35f89740 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:53.312506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 741/1491] 10.222.0.26 () {54 vars in 927 bytes} [Tue Oct 4 08:40:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:40:53.317698 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cdf2b-60a0-4e1f-92b8-47674018b36d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:53.318119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cdf2b-60a0-4e1f-92b8-47674018b36d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:53.318268 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cdf2b-60a0-4e1f-92b8-47674018b36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:53.383379 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-1b1cdf2b-60a0-4e1f-92b8-47674018b36d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:40:53.384587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1b1cdf2b-60a0-4e1f-92b8-47674018b36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:53.481230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 751/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Tue Oct 4 08:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:40:53.772846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a0134-8ac8-4dd7-97ae-499b35f89740 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:40:53.774430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-098a0134-8ac8-4dd7-97ae-499b35f89740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:53.876178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a0134-8ac8-4dd7-97ae-499b35f89740 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:40:53.950071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab802d-4766-4981-90fc-cd93e9f495c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:40:53.950596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab802d-4766-4981-90fc-cd93e9f495c2 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:40:53.950876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab802d-4766-4981-90fc-cd93e9f495c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:40:53.951153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab802d-4766-4981-90fc-cd93e9f495c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:40:53.952766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-63ab802d-4766-4981-90fc-cd93e9f495c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:40:53.961127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:40:54.060377 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab802d-4766-4981-90fc-cd93e9f495c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:40:54.061888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab802d-4766-4981-90fc-cd93e9f495c2 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:40:54.079862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 742/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:40:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:41:03.479737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=82 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:41:04.078072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=50 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:06.218986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99c286-c808-4b0a-bb43-1f9735ef07fc None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:06.219588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99c286-c808-4b0a-bb43-1f9735ef07fc None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:06.219588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99c286-c808-4b0a-bb43-1f9735ef07fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:06.294001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6f99c286-c808-4b0a-bb43-1f9735ef07fc None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:42:06.295270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6f99c286-c808-4b0a-bb43-1f9735ef07fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:06.390129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 752/1494] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:42:06.398049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe04e5-5b4d-4cd9-b58b-e47655790e78 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:06.398524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe04e5-5b4d-4cd9-b58b-e47655790e78 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:06.398633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe04e5-5b4d-4cd9-b58b-e47655790e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:06.473707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-9abe04e5-5b4d-4cd9-b58b-e47655790e78 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:42:06.474841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9abe04e5-5b4d-4cd9-b58b-e47655790e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:06.567737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 743/1495] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:42:06.572296 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99c286-c808-4b0a-bb43-1f9735ef07fc None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:06.573486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6f99c286-c808-4b0a-bb43-1f9735ef07fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:06.585263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99c286-c808-4b0a-bb43-1f9735ef07fc None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:06.645426 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:06.679634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6354dfa-60c4-4847-be1a-0497199c5b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:06.680239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6354dfa-60c4-4847-be1a-0497199c5b9a admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:06.680519 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6354dfa-60c4-4847-be1a-0497199c5b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:06.680789 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6354dfa-60c4-4847-be1a-0497199c5b9a admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:06.681428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6354dfa-60c4-4847-be1a-0497199c5b9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:42:06.681755 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6354dfa-60c4-4847-be1a-0497199c5b9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:06.682772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: WARNING py.warnings [None req-c6354dfa-60c4-4847-be1a-0497199c5b9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:42:06.682772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: warnings.warn(msg) Oct 04 08:42:06.682772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: Oct 04 08:42:06.683663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6354dfa-60c4-4847-be1a-0497199c5b9a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:06.695823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 753/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:42:06.704683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe04e5-5b4d-4cd9-b58b-e47655790e78 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:06.713673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe04e5-5b4d-4cd9-b58b-e47655790e78 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:06.774679 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:06.808154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cec86-26b4-4ee3-84d0-4b66d25a6969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:06.808782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cec86-26b4-4ee3-84d0-4b66d25a6969 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:06.809064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cec86-26b4-4ee3-84d0-4b66d25a6969 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:06.809343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cec86-26b4-4ee3-84d0-4b66d25a6969 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:06.810027 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cec86-26b4-4ee3-84d0-4b66d25a6969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:06.812348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: WARNING py.warnings [None req-227cec86-26b4-4ee3-84d0-4b66d25a6969 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:42:06.812348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: warnings.warn(msg) Oct 04 08:42:06.812348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: Oct 04 08:42:06.813594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cec86-26b4-4ee3-84d0-4b66d25a6969 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:06.879542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 744/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:42:06 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:42:06.883840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6354dfa-60c4-4847-be1a-0497199c5b9a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:06.885401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c6354dfa-60c4-4847-be1a-0497199c5b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:06.920180 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6354dfa-60c4-4847-be1a-0497199c5b9a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:07.011463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc1700-23d1-4114-bf29-9f2c6c284efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:07.012056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc1700-23d1-4114-bf29-9f2c6c284efa admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.012328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc1700-23d1-4114-bf29-9f2c6c284efa admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.012592 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc1700-23d1-4114-bf29-9f2c6c284efa admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.013202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc1700-23d1-4114-bf29-9f2c6c284efa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:07.014788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: WARNING py.warnings [None req-d8cc1700-23d1-4114-bf29-9f2c6c284efa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:42:07.014788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: warnings.warn(msg) Oct 04 08:42:07.014788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: Oct 04 08:42:07.015992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc1700-23d1-4114-bf29-9f2c6c284efa admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:07.056729 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 754/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:42:06 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:42:07.061022 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cec86-26b4-4ee3-84d0-4b66d25a6969 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:07.062586 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-227cec86-26b4-4ee3-84d0-4b66d25a6969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:07.126532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cec86-26b4-4ee3-84d0-4b66d25a6969 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:07.209667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:07.221583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8bc8-cc84-4ffd-a0ce-4bf816f30e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:07.222475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8bc8-cc84-4ffd-a0ce-4bf816f30e20 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.222752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8bc8-cc84-4ffd-a0ce-4bf816f30e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.223021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d8bc8-cc84-4ffd-a0ce-4bf816f30e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.223659 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d8bc8-cc84-4ffd-a0ce-4bf816f30e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:07.224664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: WARNING py.warnings [None req-ac9d8bc8-cc84-4ffd-a0ce-4bf816f30e20 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:42:07.224664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: warnings.warn(msg) Oct 04 08:42:07.224664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: Oct 04 08:42:07.225547 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d8bc8-cc84-4ffd-a0ce-4bf816f30e20 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:07.237899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 745/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:42:07.241913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc1700-23d1-4114-bf29-9f2c6c284efa admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:07.243485 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d8cc1700-23d1-4114-bf29-9f2c6c284efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:07.254804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc1700-23d1-4114-bf29-9f2c6c284efa admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:07.280381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:07.350615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bd8ee-7ec3-452e-9161-c498f22c80d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:07.351144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bd8ee-7ec3-452e-9161-c498f22c80d5 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.351425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bd8ee-7ec3-452e-9161-c498f22c80d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.351696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bd8ee-7ec3-452e-9161-c498f22c80d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.352363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010bd8ee-7ec3-452e-9161-c498f22c80d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:07.353381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: WARNING py.warnings [None req-010bd8ee-7ec3-452e-9161-c498f22c80d5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:42:07.353381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: warnings.warn(msg) Oct 04 08:42:07.353381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: Oct 04 08:42:07.354294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010bd8ee-7ec3-452e-9161-c498f22c80d5 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:07.366330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 755/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:42:07.370482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8bc8-cc84-4ffd-a0ce-4bf816f30e20 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:07.372050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ac9d8bc8-cc84-4ffd-a0ce-4bf816f30e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:07.406452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d8bc8-cc84-4ffd-a0ce-4bf816f30e20 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:07.501399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26ed9f-3ee8-458b-a3d2-bec3b1ec5f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:07.502075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26ed9f-3ee8-458b-a3d2-bec3b1ec5f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.502360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26ed9f-3ee8-458b-a3d2-bec3b1ec5f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.502628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26ed9f-3ee8-458b-a3d2-bec3b1ec5f1a admin admin] PATH_INFO: `/v3/projects/6894fa2b5a0e4e7cbab3818b05b07c35/users/30390800106b45f9abdea6bea6becc1b/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.524649 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26ed9f-3ee8-458b-a3d2-bec3b1ec5f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6894fa2b5a0e4e7cbab3818b05b07c35, user_id=30390800106b45f9abdea6bea6becc1b, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:07.525657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: WARNING py.warnings [None req-ae26ed9f-3ee8-458b-a3d2-bec3b1ec5f1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:42:07.525657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: warnings.warn(msg) Oct 04 08:42:07.525657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: Oct 04 08:42:07.527612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26ed9f-3ee8-458b-a3d2-bec3b1ec5f1a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:07.550792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 746/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:42:07 2022] PUT /identity/v3/projects/6894fa2b5a0e4e7cbab3818b05b07c35/users/30390800106b45f9abdea6bea6becc1b/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:42:07.563394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d1117-5cd6-433e-a55a-ff6aabc82155 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.563680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d1117-5cd6-433e-a55a-ff6aabc82155 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.563954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d1117-5cd6-433e-a55a-ff6aabc82155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.621981 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-310d1117-5cd6-433e-a55a-ff6aabc82155 None None] MFA Rules not processed for user `30390800106b45f9abdea6bea6becc1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:42:07.623746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-310d1117-5cd6-433e-a55a-ff6aabc82155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:07.702312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 756/1502] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:42:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:42:07.715458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48923a-e26b-49a1-8cf1-22a42b6e37b2 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.715803 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48923a-e26b-49a1-8cf1-22a42b6e37b2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.716080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48923a-e26b-49a1-8cf1-22a42b6e37b2 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.717443 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 747/1503] 10.222.0.26 () {54 vars in 963 bytes} [Tue Oct 4 08:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:42:07.722577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79edfaf-62fd-46aa-b51e-4e500f8388fb None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:07.722863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79edfaf-62fd-46aa-b51e-4e500f8388fb None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:07.723136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79edfaf-62fd-46aa-b51e-4e500f8388fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:07.754864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:07.798498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d79edfaf-62fd-46aa-b51e-4e500f8388fb None None] MFA Rules not processed for user `6f224ed840244faea117fe97c29c26d1`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:42:07.799726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d79edfaf-62fd-46aa-b51e-4e500f8388fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:07.874302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 757/1504] 10.222.0.26 () {58 vars in 1060 bytes} [Tue Oct 4 08:42:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:42:07.882088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48923a-e26b-49a1-8cf1-22a42b6e37b2 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:07.883597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0c48923a-e26b-49a1-8cf1-22a42b6e37b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:07.932641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:07.986707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48923a-e26b-49a1-8cf1-22a42b6e37b2 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:08.060316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e3c54-e19d-43af-b8ab-d1a92f04ef37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:08.060915 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e3c54-e19d-43af-b8ab-d1a92f04ef37 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:08.061188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e3c54-e19d-43af-b8ab-d1a92f04ef37 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:08.061453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e3c54-e19d-43af-b8ab-d1a92f04ef37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:08.063049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-044e3c54-e19d-43af-b8ab-d1a92f04ef37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:08.158644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e3c54-e19d-43af-b8ab-d1a92f04ef37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:08.160028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e3c54-e19d-43af-b8ab-d1a92f04ef37 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:08.163420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:08.178886 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 748/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:42:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:42:08.226641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d37199-1d27-427c-ba87-e2e5e4ca36d1 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:08.226926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d37199-1d27-427c-ba87-e2e5e4ca36d1 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:08.227202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d37199-1d27-427c-ba87-e2e5e4ca36d1 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:08.228477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 758/1506] 10.222.0.26 () {52 vars in 830 bytes} [Tue Oct 4 08:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:42:08.232849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e3c54-e19d-43af-b8ab-d1a92f04ef37 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:08.234130 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-044e3c54-e19d-43af-b8ab-d1a92f04ef37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:08.329855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e3c54-e19d-43af-b8ab-d1a92f04ef37 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:08.422708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:08.423269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:08.423535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:08.423802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] PATH_INFO: `/v3/users/30390800106b45f9abdea6bea6becc1b` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:08.436436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30390800106b45f9abdea6bea6becc1b)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:08.437454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: WARNING py.warnings [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:42:08.437454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: warnings.warn(msg) Oct 04 08:42:08.437454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: Oct 04 08:42:08.438696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:08.448035 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:08.502847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30390800106b45f9abdea6bea6becc1b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:42:08.503378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30390800106b45f9abdea6bea6becc1b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:42:08.525680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30390800106b45f9abdea6bea6becc1b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:42:08.539794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30390800106b45f9abdea6bea6becc1b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:42:08.540252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30390800106b45f9abdea6bea6becc1b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:42:08.554024 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 749/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:42:08 2022] DELETE /identity/v3/users/30390800106b45f9abdea6bea6becc1b => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:42:08.558534 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c91f99-c6c0-49b9-a685-cd6df78ac14a None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:08.559025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c91f99-c6c0-49b9-a685-cd6df78ac14a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:08.559129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c91f99-c6c0-49b9-a685-cd6df78ac14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:08.616142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b9c91f99-c6c0-49b9-a685-cd6df78ac14a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:42:08.617339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b9c91f99-c6c0-49b9-a685-cd6df78ac14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:08.692434 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:08.705999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 759/1508] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:42:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:42:08.716289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:08.718144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:08.816340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d4f22-385a-4c89-8c1e-ee5fcbd2ef52 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:08.868558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:08.889245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6c703-d0f8-4157-b0d8-40da15fe1170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:08.889873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6c703-d0f8-4157-b0d8-40da15fe1170 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:08.890151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6c703-d0f8-4157-b0d8-40da15fe1170 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:08.890417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6c703-d0f8-4157-b0d8-40da15fe1170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:08.891977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-37e6c703-d0f8-4157-b0d8-40da15fe1170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:08.995717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6c703-d0f8-4157-b0d8-40da15fe1170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:08.997095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6c703-d0f8-4157-b0d8-40da15fe1170 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:09.016054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 750/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:42:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:42:09.453202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c91f99-c6c0-49b9-a685-cd6df78ac14a None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:42:09.454504 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b9c91f99-c6c0-49b9-a685-cd6df78ac14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:42:09.466079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c91f99-c6c0-49b9-a685-cd6df78ac14a None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:42:09.562559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:42:09.563236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:42:09.563452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:42:09.563722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] PATH_INFO: `/v3/projects/6894fa2b5a0e4e7cbab3818b05b07c35` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:42:09.574153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6894fa2b5a0e4e7cbab3818b05b07c35)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:42:09.575176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: WARNING py.warnings [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:42:09.575176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: warnings.warn(msg) Oct 04 08:42:09.575176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: Oct 04 08:42:09.576400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:42:09.634166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6894fa2b5a0e4e7cbab3818b05b07c35', 'request_id': 'req-2e30f154-52e3-48f4-a5d0-f57be9beae7c'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:42:09.646844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6894fa2b5a0e4e7cbab3818b05b07c35', 'request_id': 'req-2e30f154-52e3-48f4-a5d0-f57be9beae7c'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:42:09.652900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6894fa2b5a0e4e7cbab3818b05b07c35', 'request_id': 'req-2e30f154-52e3-48f4-a5d0-f57be9beae7c'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:42:09.679658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] The token cache is being invalidate because project 6894fa2b5a0e4e7cbab3818b05b07c35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:42:09.679944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:42:09.680403 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2e30f154-52e3-48f4-a5d0-f57be9beae7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:42:09.681835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 760/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:42:09 2022] DELETE /identity/v3/projects/6894fa2b5a0e4e7cbab3818b05b07c35 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:42:19.013828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=62 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:42:19.677615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:40.248330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d59185-5837-4c00-a5e1-d84ea6529d07 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:40.248899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d59185-5837-4c00-a5e1-d84ea6529d07 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:40.248899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d59185-5837-4c00-a5e1-d84ea6529d07 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:40.250299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:40.255038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1891eed-5983-4661-9f86-b2c58c5af715 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:40.255455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1891eed-5983-4661-9f86-b2c58c5af715 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:40.255602 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1891eed-5983-4661-9f86-b2c58c5af715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:40.336252 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-a1891eed-5983-4661-9f86-b2c58c5af715 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:40.337381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a1891eed-5983-4661-9f86-b2c58c5af715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:40.438466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:40.592597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d59185-5837-4c00-a5e1-d84ea6529d07 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:40.594776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-21d59185-5837-4c00-a5e1-d84ea6529d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:40.719316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d59185-5837-4c00-a5e1-d84ea6529d07 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:40.725031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f8998-1c23-4ea7-ba0d-add12687ff8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:40.725637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f8998-1c23-4ea7-ba0d-add12687ff8b service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:40.725950 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f8998-1c23-4ea7-ba0d-add12687ff8b service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:40.726223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f8998-1c23-4ea7-ba0d-add12687ff8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:40.734150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f8998-1c23-4ea7-ba0d-add12687ff8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:40.735722 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f8998-1c23-4ea7-ba0d-add12687ff8b service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:40.745381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 752/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:41.814441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896e126-b777-4f85-b04d-6a091a513c3d None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:41.815022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896e126-b777-4f85-b04d-6a091a513c3d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:41.815022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896e126-b777-4f85-b04d-6a091a513c3d None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:41.816324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:41.822327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ab68a-7b76-41a2-9d22-e565898ec204 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:41.822758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ab68a-7b76-41a2-9d22-e565898ec204 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:41.823410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ab68a-7b76-41a2-9d22-e565898ec204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:41.843182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-ee1ab68a-7b76-41a2-9d22-e565898ec204 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:41.844359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ee1ab68a-7b76-41a2-9d22-e565898ec204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:41.852529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:42.005258 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896e126-b777-4f85-b04d-6a091a513c3d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:42.019774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896e126-b777-4f85-b04d-6a091a513c3d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:42.025096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9bd1c-2a54-46e1-b855-567f387b9868 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:42.025714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9bd1c-2a54-46e1-b855-567f387b9868 service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:42.025998 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9bd1c-2a54-46e1-b855-567f387b9868 service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:42.026831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9bd1c-2a54-46e1-b855-567f387b9868 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:42.035114 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9bd1c-2a54-46e1-b855-567f387b9868 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:42.036731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9bd1c-2a54-46e1-b855-567f387b9868 service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:42.045963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 763/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:42.370869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f405fa-5ffa-4388-bd60-c260f663926c None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:42.371147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f405fa-5ffa-4388-bd60-c260f663926c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:42.371420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f405fa-5ffa-4388-bd60-c260f663926c None None] PATH_INFO: `/v3` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:42.372748 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 754/1517] 10.222.0.46 () {54 vars in 925 bytes} [Tue Oct 4 08:43:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:43:42.379053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41aaa65-5be1-4d39-968c-55bdcb54cf27 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:42.379339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41aaa65-5be1-4d39-968c-55bdcb54cf27 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:42.379613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41aaa65-5be1-4d39-968c-55bdcb54cf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:42.440390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-a41aaa65-5be1-4d39-968c-55bdcb54cf27 None None] MFA Rules not processed for user `abd158648914409f87ce0599843f0fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:42.441607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a41aaa65-5be1-4d39-968c-55bdcb54cf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:42.458636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:43:42.511866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 764/1518] 10.222.0.46 () {58 vars in 1015 bytes} [Tue Oct 4 08:43:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:51.520158 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844c21d-946b-4032-811f-56889107a59f None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:51.520587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844c21d-946b-4032-811f-56889107a59f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:51.520705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4844c21d-946b-4032-811f-56889107a59f None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:51.522057 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 755/1519] 10.222.0.26 () {54 vars in 961 bytes} [Tue Oct 4 08:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:51.526916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549d467-539b-4bf6-899f-2117b29cb73c None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:51.527371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549d467-539b-4bf6-899f-2117b29cb73c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:51.527481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549d467-539b-4bf6-899f-2117b29cb73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:51.546858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-f549d467-539b-4bf6-899f-2117b29cb73c None None] MFA Rules not processed for user `abd158648914409f87ce0599843f0fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:51.548068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f549d467-539b-4bf6-899f-2117b29cb73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:51.556703 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 765/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Oct 4 08:43:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:51.564602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844c21d-946b-4032-811f-56889107a59f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:51.578258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4844c21d-946b-4032-811f-56889107a59f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:51.583075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc576fe-e23c-4042-a873-5bc3a2d4e085 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:51.583656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc576fe-e23c-4042-a873-5bc3a2d4e085 service placement] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:51.583935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc576fe-e23c-4042-a873-5bc3a2d4e085 service placement] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:51.584213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc576fe-e23c-4042-a873-5bc3a2d4e085 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:51.592241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc576fe-e23c-4042-a873-5bc3a2d4e085 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:51.593847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc576fe-e23c-4042-a873-5bc3a2d4e085 service placement] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:51.603621 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 756/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:53.504646 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1ae64-d0df-4dea-b61f-cf045597fac9 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:53.505182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1ae64-d0df-4dea-b61f-cf045597fac9 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:53.505182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1ae64-d0df-4dea-b61f-cf045597fac9 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:53.506499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:53.513013 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b1f6d-e635-4637-a742-19905e08de48 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:53.513500 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b1f6d-e635-4637-a742-19905e08de48 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:53.513599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b1f6d-e635-4637-a742-19905e08de48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:53.533779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-cc7b1f6d-e635-4637-a742-19905e08de48 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:53.534993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-cc7b1f6d-e635-4637-a742-19905e08de48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:53.543187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:53.696480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1ae64-d0df-4dea-b61f-cf045597fac9 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:53.711027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1ae64-d0df-4dea-b61f-cf045597fac9 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:53.716858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbc8f4-e0c4-4fe5-8f4d-116d614f50e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:53.717500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbc8f4-e0c4-4fe5-8f4d-116d614f50e4 service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:53.717787 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbc8f4-e0c4-4fe5-8f4d-116d614f50e4 service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:53.718062 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbc8f4-e0c4-4fe5-8f4d-116d614f50e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:53.726179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbc8f4-e0c4-4fe5-8f4d-116d614f50e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:53.727749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbc8f4-e0c4-4fe5-8f4d-116d614f50e4 service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:53.737441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 767/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:54.790071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef425f5-87f2-4da8-852f-a2f13a49c118 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:54.790678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef425f5-87f2-4da8-852f-a2f13a49c118 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:54.790678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef425f5-87f2-4da8-852f-a2f13a49c118 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:54.791953 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 08:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:54.798567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb564dc8-b95b-40be-81ac-1c47c9d54a9a None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:54.799027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb564dc8-b95b-40be-81ac-1c47c9d54a9a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:54.799131 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb564dc8-b95b-40be-81ac-1c47c9d54a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:54.818734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-eb564dc8-b95b-40be-81ac-1c47c9d54a9a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:54.819926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-eb564dc8-b95b-40be-81ac-1c47c9d54a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:54.827704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 08:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:54.980824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef425f5-87f2-4da8-852f-a2f13a49c118 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:54.995610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef425f5-87f2-4da8-852f-a2f13a49c118 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:55.000963 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f99fe3-407f-4e97-97bc-a5fa88934459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:55.001581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f99fe3-407f-4e97-97bc-a5fa88934459 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:55.001887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f99fe3-407f-4e97-97bc-a5fa88934459 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:55.002161 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f99fe3-407f-4e97-97bc-a5fa88934459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:55.010008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f99fe3-407f-4e97-97bc-a5fa88934459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:55.011580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f99fe3-407f-4e97-97bc-a5fa88934459 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:55.020786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 759/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:43:59.052767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc9d3-3e5e-47f4-91c4-a9ea082b9b7b None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:59.053348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc9d3-3e5e-47f4-91c4-a9ea082b9b7b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:59.053348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bc9d3-3e5e-47f4-91c4-a9ea082b9b7b None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:59.054711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:43:59.060075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725676b7-e6ce-4bbb-a020-ede26791d485 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:59.060529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725676b7-e6ce-4bbb-a020-ede26791d485 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:59.060639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725676b7-e6ce-4bbb-a020-ede26791d485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:59.080390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-725676b7-e6ce-4bbb-a020-ede26791d485 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:43:59.081585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-725676b7-e6ce-4bbb-a020-ede26791d485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:59.089214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:43:59.381668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bc9d3-3e5e-47f4-91c4-a9ea082b9b7b None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:43:59.383413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-814bc9d3-3e5e-47f4-91c4-a9ea082b9b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:43:59.460904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bc9d3-3e5e-47f4-91c4-a9ea082b9b7b None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:43:59.464455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4835f-e959-4197-a8bb-8df6c2a02eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:43:59.465057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4835f-e959-4197-a8bb-8df6c2a02eed service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:43:59.465340 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4835f-e959-4197-a8bb-8df6c2a02eed service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:43:59.465645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4835f-e959-4197-a8bb-8df6c2a02eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:43:59.474746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4835f-e959-4197-a8bb-8df6c2a02eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:43:59.476154 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4835f-e959-4197-a8bb-8df6c2a02eed service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:43:59.485350 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 770/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:01.595706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374114a0-6f80-4e33-af34-5f4a3204e3b8 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:01.596305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374114a0-6f80-4e33-af34-5f4a3204e3b8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:01.596305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374114a0-6f80-4e33-af34-5f4a3204e3b8 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:01.597971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:44:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:01.603313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bffa86-e101-4ef1-8109-36c328f4fbda None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:01.603743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bffa86-e101-4ef1-8109-36c328f4fbda None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:01.603853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bffa86-e101-4ef1-8109-36c328f4fbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:01.623871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-57bffa86-e101-4ef1-8109-36c328f4fbda None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:01.625054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-57bffa86-e101-4ef1-8109-36c328f4fbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:01.632652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:44:01.916260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374114a0-6f80-4e33-af34-5f4a3204e3b8 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:01.927405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374114a0-6f80-4e33-af34-5f4a3204e3b8 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:01.930822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9070aee-9b64-4474-81e7-3a18fe2c23fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:01.931405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9070aee-9b64-4474-81e7-3a18fe2c23fd service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:01.931689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9070aee-9b64-4474-81e7-3a18fe2c23fd service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:01.931959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9070aee-9b64-4474-81e7-3a18fe2c23fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:01.940479 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9070aee-9b64-4474-81e7-3a18fe2c23fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:01.941912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9070aee-9b64-4474-81e7-3a18fe2c23fd service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:01.950949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 762/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:05.627684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fe261-a682-4a00-8660-ec154b78d2a5 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:05.628268 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fe261-a682-4a00-8660-ec154b78d2a5 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:05.628268 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fe261-a682-4a00-8660-ec154b78d2a5 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:05.629577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:05.634912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e6289-814a-48c0-9053-0a21a3601efa None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:05.635309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e6289-814a-48c0-9053-0a21a3601efa None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:05.635461 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e6289-814a-48c0-9053-0a21a3601efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:05.656157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-674e6289-814a-48c0-9053-0a21a3601efa None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:05.657373 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-674e6289-814a-48c0-9053-0a21a3601efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:05.665512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:05.964506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9fe261-a682-4a00-8660-ec154b78d2a5 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:05.966253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2b9fe261-a682-4a00-8660-ec154b78d2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:06.030248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:06.044711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9fe261-a682-4a00-8660-ec154b78d2a5 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:06.048338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b09673-afc2-4e68-af73-bf26d74b23fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:06.048939 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b09673-afc2-4e68-af73-bf26d74b23fe service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:06.049221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b09673-afc2-4e68-af73-bf26d74b23fe service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:06.049494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b09673-afc2-4e68-af73-bf26d74b23fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:06.058282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b09673-afc2-4e68-af73-bf26d74b23fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:06.060334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b09673-afc2-4e68-af73-bf26d74b23fe service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:06.069611 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 773/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:08.187423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7148775-ebb1-41aa-a786-a727a4d0f52c None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:08.188001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7148775-ebb1-41aa-a786-a727a4d0f52c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:08.188107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7148775-ebb1-41aa-a786-a727a4d0f52c None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:08.189505 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 08:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:44:08.194936 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ed4a3-9c0e-46b7-91ae-e66e6f4de81a None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:08.195388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ed4a3-9c0e-46b7-91ae-e66e6f4de81a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:08.195505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494ed4a3-9c0e-46b7-91ae-e66e6f4de81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:08.214512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-494ed4a3-9c0e-46b7-91ae-e66e6f4de81a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:44:08.215694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-494ed4a3-9c0e-46b7-91ae-e66e6f4de81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:08.223619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 08:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:44:08.533595 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7148775-ebb1-41aa-a786-a727a4d0f52c None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:44:08.535287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d7148775-ebb1-41aa-a786-a727a4d0f52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:44:08.549005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7148775-ebb1-41aa-a786-a727a4d0f52c None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:44:08.552445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6ff36-1e80-45dc-85fd-8ab08066bede service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:44:08.553031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6ff36-1e80-45dc-85fd-8ab08066bede service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:44:08.553305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6ff36-1e80-45dc-85fd-8ab08066bede service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:44:08.553571 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6ff36-1e80-45dc-85fd-8ab08066bede service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:44:08.562177 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6ff36-1e80-45dc-85fd-8ab08066bede service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:44:08.563585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6ff36-1e80-45dc-85fd-8ab08066bede service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:44:08.572940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 765/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:44:18.209821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=11 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:44:18.568892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:33.744824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e5d7b-51e5-406a-9cea-84de0586ac71 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:33.745466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e5d7b-51e5-406a-9cea-84de0586ac71 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:33.745466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e5d7b-51e5-406a-9cea-84de0586ac71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:33.771509 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d75e5d7b-51e5-406a-9cea-84de0586ac71 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:33.773096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d75e5d7b-51e5-406a-9cea-84de0586ac71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:33.788208 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 775/1540] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:45:33.796971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f17c7-f48d-4989-9ab8-bf4f43cd4a8e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:33.797536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f17c7-f48d-4989-9ab8-bf4f43cd4a8e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:33.797536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f17c7-f48d-4989-9ab8-bf4f43cd4a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:33.825804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-ff4f17c7-f48d-4989-9ab8-bf4f43cd4a8e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:33.827089 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ff4f17c7-f48d-4989-9ab8-bf4f43cd4a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:33.840324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 766/1541] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:45:33.845100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e5d7b-51e5-406a-9cea-84de0586ac71 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:33.859725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e5d7b-51e5-406a-9cea-84de0586ac71 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:33.867680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2fccc-c81c-4587-9613-8ca3518fd35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:33.868734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2fccc-c81c-4587-9613-8ca3518fd35b admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:33.870023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2fccc-c81c-4587-9613-8ca3518fd35b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:33.870518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2fccc-c81c-4587-9613-8ca3518fd35b admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:33.871412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2fccc-c81c-4587-9613-8ca3518fd35b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:45:33.872005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2fccc-c81c-4587-9613-8ca3518fd35b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:33.874287 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2fccc-c81c-4587-9613-8ca3518fd35b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:33.897742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 776/1542] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:45:33.906952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f17c7-f48d-4989-9ab8-bf4f43cd4a8e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:33.923625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f17c7-f48d-4989-9ab8-bf4f43cd4a8e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:33.930882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dc8b0-ddf9-4cbc-8288-77e154ef081d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:33.932065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dc8b0-ddf9-4cbc-8288-77e154ef081d admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:33.932360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dc8b0-ddf9-4cbc-8288-77e154ef081d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:33.932633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5dc8b0-ddf9-4cbc-8288-77e154ef081d admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:33.933316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5dc8b0-ddf9-4cbc-8288-77e154ef081d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:33.935447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5dc8b0-ddf9-4cbc-8288-77e154ef081d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:33.954243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79017fb6-ac3a-44be-a76d-d1079f6a3365 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:33.955114 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79017fb6-ac3a-44be-a76d-d1079f6a3365 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:33.955655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79017fb6-ac3a-44be-a76d-d1079f6a3365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:33.989042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-79017fb6-ac3a-44be-a76d-d1079f6a3365 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:33.990798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-79017fb6-ac3a-44be-a76d-d1079f6a3365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.004724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 767/1543] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:33 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:34.009573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dc8b0-ddf9-4cbc-8288-77e154ef081d admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.011211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 777/1544] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.020006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cb819-bef4-43f0-80ca-09bb760db7a2 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.020625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cb819-bef4-43f0-80ca-09bb760db7a2 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.021113 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cb819-bef4-43f0-80ca-09bb760db7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.061440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-e42cb819-bef4-43f0-80ca-09bb760db7a2 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.063292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e42cb819-bef4-43f0-80ca-09bb760db7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.102066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5dc8b0-ddf9-4cbc-8288-77e154ef081d admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.118431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd513e26-ac25-4734-b547-fd35353a71e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.119092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd513e26-ac25-4734-b547-fd35353a71e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.119378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd513e26-ac25-4734-b547-fd35353a71e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.119662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd513e26-ac25-4734-b547-fd35353a71e8 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.120314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd513e26-ac25-4734-b547-fd35353a71e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.121341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: WARNING py.warnings [None req-fd513e26-ac25-4734-b547-fd35353a71e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:45:34.121341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: warnings.warn(msg) Oct 04 08:45:34.121341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: Oct 04 08:45:34.122917 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd513e26-ac25-4734-b547-fd35353a71e8 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.165041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 768/1545] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:34.169608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd513e26-ac25-4734-b547-fd35353a71e8 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.174395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 778/1546] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:45:34.178606 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cb819-bef4-43f0-80ca-09bb760db7a2 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.187365 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd513e26-ac25-4734-b547-fd35353a71e8 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.203504 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a72fed-1b26-44aa-8838-aad687ab787b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.204283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a72fed-1b26-44aa-8838-aad687ab787b admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.204566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a72fed-1b26-44aa-8838-aad687ab787b admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.204818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cb819-bef4-43f0-80ca-09bb760db7a2 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.204950 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a72fed-1b26-44aa-8838-aad687ab787b admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.205637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a72fed-1b26-44aa-8838-aad687ab787b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.207272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a72fed-1b26-44aa-8838-aad687ab787b admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.226452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7d1cf-e4a0-43a6-9058-d4086184d024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.227087 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7d1cf-e4a0-43a6-9058-d4086184d024 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.227371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7d1cf-e4a0-43a6-9058-d4086184d024 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.227645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7d1cf-e4a0-43a6-9058-d4086184d024 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.228622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7d1cf-e4a0-43a6-9058-d4086184d024 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:45:34.228934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7d1cf-e4a0-43a6-9058-d4086184d024 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.230578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7d1cf-e4a0-43a6-9058-d4086184d024 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.231342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 769/1547] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:34.232996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227226df-b11c-4023-bd23-5d4d41138560 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.233680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227226df-b11c-4023-bd23-5d4d41138560 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.234568 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227226df-b11c-4023-bd23-5d4d41138560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.252003 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 779/1548] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:45:34.252868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7d1cf-e4a0-43a6-9058-d4086184d024 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.272522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7d1cf-e4a0-43a6-9058-d4086184d024 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.282769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-227226df-b11c-4023-bd23-5d4d41138560 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.284641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-227226df-b11c-4023-bd23-5d4d41138560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.286500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04491b-c4d7-4c80-ac5d-94e73c9932de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.287134 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04491b-c4d7-4c80-ac5d-94e73c9932de admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.287417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04491b-c4d7-4c80-ac5d-94e73c9932de admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.287693 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04491b-c4d7-4c80-ac5d-94e73c9932de admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.288345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04491b-c4d7-4c80-ac5d-94e73c9932de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.290082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04491b-c4d7-4c80-ac5d-94e73c9932de admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.300025 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 770/1549] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.301164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227226df-b11c-4023-bd23-5d4d41138560 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.309678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 780/1550] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:34.315966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04491b-c4d7-4c80-ac5d-94e73c9932de admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.318812 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227226df-b11c-4023-bd23-5d4d41138560 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.331160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09079916-e90e-4b75-b5e3-745004f78836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.332162 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09079916-e90e-4b75-b5e3-745004f78836 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.332639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09079916-e90e-4b75-b5e3-745004f78836 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.333123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09079916-e90e-4b75-b5e3-745004f78836 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.334207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09079916-e90e-4b75-b5e3-745004f78836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.335628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04491b-c4d7-4c80-ac5d-94e73c9932de admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.336940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09079916-e90e-4b75-b5e3-745004f78836 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.344556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874995e4-8c2f-4b8e-8a81-9900e65e9d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.345237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874995e4-8c2f-4b8e-8a81-9900e65e9d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.346069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874995e4-8c2f-4b8e-8a81-9900e65e9d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.346358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874995e4-8c2f-4b8e-8a81-9900e65e9d96 admin admin] PATH_INFO: `/v3/projects/fcafb472d06f4b5eb1ca07e95611b88a/users/44d19bdc64da4d72ab5e37aa53188b5c/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.364435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874995e4-8c2f-4b8e-8a81-9900e65e9d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcafb472d06f4b5eb1ca07e95611b88a, user_id=44d19bdc64da4d72ab5e37aa53188b5c, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.365657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: WARNING py.warnings [None req-874995e4-8c2f-4b8e-8a81-9900e65e9d96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:45:34.365657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: warnings.warn(msg) Oct 04 08:45:34.365657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: Oct 04 08:45:34.367885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874995e4-8c2f-4b8e-8a81-9900e65e9d96 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.373659 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 771/1551] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/projects => generated 389 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:34.374959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de849498-7fec-403c-a1d4-478c50c4773b None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.375243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de849498-7fec-403c-a1d4-478c50c4773b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.375666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de849498-7fec-403c-a1d4-478c50c4773b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.383517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 781/1552] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:34 2022] PUT /identity/v3/projects/fcafb472d06f4b5eb1ca07e95611b88a/users/44d19bdc64da4d72ab5e37aa53188b5c/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:34.385843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbc5a4-da34-42ae-80f6-9feac2f70e08 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.386129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbc5a4-da34-42ae-80f6-9feac2f70e08 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.386408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbc5a4-da34-42ae-80f6-9feac2f70e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.400502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-de849498-7fec-403c-a1d4-478c50c4773b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.401718 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-de849498-7fec-403c-a1d4-478c50c4773b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.405712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-94bbc5a4-da34-42ae-80f6-9feac2f70e08 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.407080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-94bbc5a4-da34-42ae-80f6-9feac2f70e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.461760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 772/1553] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.462959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac1cb8-0490-4d8b-a589-752da29ea3fe None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.463246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac1cb8-0490-4d8b-a589-752da29ea3fe None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.463527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac1cb8-0490-4d8b-a589-752da29ea3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.466442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 782/1554] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.467523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193e7d6-3e63-4442-b503-34432113f8a5 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.467805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193e7d6-3e63-4442-b503-34432113f8a5 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.468083 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193e7d6-3e63-4442-b503-34432113f8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.487424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-8193e7d6-3e63-4442-b503-34432113f8a5 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.487641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-ffac1cb8-0490-4d8b-a589-752da29ea3fe None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.488630 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8193e7d6-3e63-4442-b503-34432113f8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.488779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ffac1cb8-0490-4d8b-a589-752da29ea3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.497092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 783/1555] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.497231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 773/1556] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.504834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c931a-ac62-406f-9f18-acbd95c32012 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.505032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9b1b0-62d8-43dc-bb3f-b31cb442529f None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.505149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c931a-ac62-406f-9f18-acbd95c32012 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.505266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9b1b0-62d8-43dc-bb3f-b31cb442529f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.505367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c931a-ac62-406f-9f18-acbd95c32012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.505458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9b1b0-62d8-43dc-bb3f-b31cb442529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.516649 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:34.522938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-8fe9b1b0-62d8-43dc-bb3f-b31cb442529f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.523247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-771c931a-ac62-406f-9f18-acbd95c32012 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.524470 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-771c931a-ac62-406f-9f18-acbd95c32012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.524744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8fe9b1b0-62d8-43dc-bb3f-b31cb442529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.532424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 784/1557] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.532852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 774/1558] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.533307 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c931a-ac62-406f-9f18-acbd95c32012 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.538899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef38cf-5996-469e-a622-9237cd33444b None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.539290 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef38cf-5996-469e-a622-9237cd33444b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.539713 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef38cf-5996-469e-a622-9237cd33444b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.545168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c931a-ac62-406f-9f18-acbd95c32012 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.549791 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af82b43-a922-40a2-a0e9-de793ae32f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.550420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af82b43-a922-40a2-a0e9-de793ae32f04 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.550699 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af82b43-a922-40a2-a0e9-de793ae32f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.550973 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af82b43-a922-40a2-a0e9-de793ae32f04 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.551599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af82b43-a922-40a2-a0e9-de793ae32f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.553530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af82b43-a922-40a2-a0e9-de793ae32f04 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.565288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:34.569813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c2ef38cf-5996-469e-a622-9237cd33444b None None] MFA Rules not processed for user `44d19bdc64da4d72ab5e37aa53188b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.571654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c2ef38cf-5996-469e-a622-9237cd33444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.578071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 785/1559] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/users => generated 295 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:34.579292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4e2f3-a173-4ec4-b803-e907554d684d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.579578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4e2f3-a173-4ec4-b803-e907554d684d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.579853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4e2f3-a173-4ec4-b803-e907554d684d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.599116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-fcc4e2f3-a173-4ec4-b803-e907554d684d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.600321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fcc4e2f3-a173-4ec4-b803-e907554d684d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.608163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 786/1560] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.608930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4e2f3-a173-4ec4-b803-e907554d684d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.618267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4e2f3-a173-4ec4-b803-e907554d684d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.623479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2388995-02df-4e63-b71b-2bcc8a29838c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.624063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2388995-02df-4e63-b71b-2bcc8a29838c admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.624343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2388995-02df-4e63-b71b-2bcc8a29838c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.624619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2388995-02df-4e63-b71b-2bcc8a29838c admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.625259 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2388995-02df-4e63-b71b-2bcc8a29838c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:45:34.625567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2388995-02df-4e63-b71b-2bcc8a29838c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.627206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2388995-02df-4e63-b71b-2bcc8a29838c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.641639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 787/1561] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:45:34.644791 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012c2c8-1413-467f-b7e3-be9eaa4b3e07 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.645078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012c2c8-1413-467f-b7e3-be9eaa4b3e07 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.645353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012c2c8-1413-467f-b7e3-be9eaa4b3e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.653934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 775/1562] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.656135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5b944-d876-4522-b2ee-bfa269566908 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.657574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5b944-d876-4522-b2ee-bfa269566908 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.657574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5b944-d876-4522-b2ee-bfa269566908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.666383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d012c2c8-1413-467f-b7e3-be9eaa4b3e07 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.667606 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d012c2c8-1413-467f-b7e3-be9eaa4b3e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.675781 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 788/1563] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.679595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18dcd0-e95b-49a9-9ade-384115bbff12 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.679595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18dcd0-e95b-49a9-9ade-384115bbff12 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.679595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18dcd0-e95b-49a9-9ade-384115bbff12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.679850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-4be5b944-d876-4522-b2ee-bfa269566908 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.679850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4be5b944-d876-4522-b2ee-bfa269566908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.688008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 776/1564] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.689197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe59e4-50a1-4ef5-9c90-a0ec9caec840 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.689481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe59e4-50a1-4ef5-9c90-a0ec9caec840 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.689778 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe59e4-50a1-4ef5-9c90-a0ec9caec840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.698448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6a18dcd0-e95b-49a9-9ade-384115bbff12 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.699634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6a18dcd0-e95b-49a9-9ade-384115bbff12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.707340 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 789/1565] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.710949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18dcd0-e95b-49a9-9ade-384115bbff12 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.711039 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-52fe59e4-50a1-4ef5-9c90-a0ec9caec840 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:34.711039 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-52fe59e4-50a1-4ef5-9c90-a0ec9caec840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:34.718509 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 777/1566] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:34.719337 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe59e4-50a1-4ef5-9c90-a0ec9caec840 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.725037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18dcd0-e95b-49a9-9ade-384115bbff12 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.729077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe59e4-50a1-4ef5-9c90-a0ec9caec840 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.729739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871694e-aa64-482a-bca8-f32ef7e46fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.730380 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871694e-aa64-482a-bca8-f32ef7e46fed admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.730661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871694e-aa64-482a-bca8-f32ef7e46fed admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.730931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871694e-aa64-482a-bca8-f32ef7e46fed admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.731575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871694e-aa64-482a-bca8-f32ef7e46fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.733544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871694e-aa64-482a-bca8-f32ef7e46fed admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.734851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc49cb-bb0c-44a4-9304-28d48c87b051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.735459 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc49cb-bb0c-44a4-9304-28d48c87b051 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.735995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc49cb-bb0c-44a4-9304-28d48c87b051 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.735995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc49cb-bb0c-44a4-9304-28d48c87b051 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.736638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc49cb-bb0c-44a4-9304-28d48c87b051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:45:34.736969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc49cb-bb0c-44a4-9304-28d48c87b051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.738273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: WARNING py.warnings [None req-aefc49cb-bb0c-44a4-9304-28d48c87b051 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:45:34.738273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: warnings.warn(msg) Oct 04 08:45:34.738273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: Oct 04 08:45:34.739182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc49cb-bb0c-44a4-9304-28d48c87b051 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.744727 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 790/1567] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:34.745352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871694e-aa64-482a-bca8-f32ef7e46fed admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.757222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 778/1568] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:45:34.757222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc49cb-bb0c-44a4-9304-28d48c87b051 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.757468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871694e-aa64-482a-bca8-f32ef7e46fed admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.761575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8f639-600f-4b0a-bfd3-7d7ba99947e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.762204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8f639-600f-4b0a-bfd3-7d7ba99947e4 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.762481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8f639-600f-4b0a-bfd3-7d7ba99947e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.762752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8f639-600f-4b0a-bfd3-7d7ba99947e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.763397 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8f639-600f-4b0a-bfd3-7d7ba99947e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.764407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: WARNING py.warnings [None req-1cb8f639-600f-4b0a-bfd3-7d7ba99947e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:45:34.764407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: warnings.warn(msg) Oct 04 08:45:34.764407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: Oct 04 08:45:34.765624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8f639-600f-4b0a-bfd3-7d7ba99947e4 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.768893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc49cb-bb0c-44a4-9304-28d48c87b051 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.773371 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac416c-6f3d-4ca0-9cf7-858aace5fd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.773994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac416c-6f3d-4ca0-9cf7-858aace5fd2c admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.774274 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac416c-6f3d-4ca0-9cf7-858aace5fd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.774545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac416c-6f3d-4ca0-9cf7-858aace5fd2c admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.775167 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac416c-6f3d-4ca0-9cf7-858aace5fd2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.777083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac416c-6f3d-4ca0-9cf7-858aace5fd2c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.795596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 791/1569] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:34.795596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8f639-600f-4b0a-bfd3-7d7ba99947e4 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.807313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 779/1570] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:34.808061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac416c-6f3d-4ca0-9cf7-858aace5fd2c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.858629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8f639-600f-4b0a-bfd3-7d7ba99947e4 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.875193 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac416c-6f3d-4ca0-9cf7-858aace5fd2c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.879868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6e982-92bd-4e59-bf88-94fa7f4c01a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.880478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6e982-92bd-4e59-bf88-94fa7f4c01a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.880758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6e982-92bd-4e59-bf88-94fa7f4c01a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.881094 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6e982-92bd-4e59-bf88-94fa7f4c01a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.881841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6e982-92bd-4e59-bf88-94fa7f4c01a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:45:34.882153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6e982-92bd-4e59-bf88-94fa7f4c01a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.883767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6e982-92bd-4e59-bf88-94fa7f4c01a3 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.896423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 780/1571] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:45:34.897237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6e982-92bd-4e59-bf88-94fa7f4c01a3 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.906574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6e982-92bd-4e59-bf88-94fa7f4c01a3 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.911253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a3b44-cf61-4b5c-b273-3c12da219587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.911829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a3b44-cf61-4b5c-b273-3c12da219587 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.912104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a3b44-cf61-4b5c-b273-3c12da219587 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.912376 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a3b44-cf61-4b5c-b273-3c12da219587 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.913003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a3b44-cf61-4b5c-b273-3c12da219587 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:45:34.913305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a3b44-cf61-4b5c-b273-3c12da219587 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.914902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a3b44-cf61-4b5c-b273-3c12da219587 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.922624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ae708-5327-4453-867a-bfb2b24b4071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.923258 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ae708-5327-4453-867a-bfb2b24b4071 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.923539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ae708-5327-4453-867a-bfb2b24b4071 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.923823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ae708-5327-4453-867a-bfb2b24b4071 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.924466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ae708-5327-4453-867a-bfb2b24b4071 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:45:34.924770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ae708-5327-4453-867a-bfb2b24b4071 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.926400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ae708-5327-4453-867a-bfb2b24b4071 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.930517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 781/1572] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:45:34.930517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a3b44-cf61-4b5c-b273-3c12da219587 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.937452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a3b44-cf61-4b5c-b273-3c12da219587 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.949209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 792/1573] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:45:34.949418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba1162-4686-43fb-a7c3-e000c95412f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.949418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba1162-4686-43fb-a7c3-e000c95412f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.949627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ae708-5327-4453-867a-bfb2b24b4071 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.949772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba1162-4686-43fb-a7c3-e000c95412f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.949772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba1162-4686-43fb-a7c3-e000c95412f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.949772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba1162-4686-43fb-a7c3-e000c95412f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:45:34.949772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba1162-4686-43fb-a7c3-e000c95412f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.949772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba1162-4686-43fb-a7c3-e000c95412f7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.953340 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ae708-5327-4453-867a-bfb2b24b4071 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.958179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8628d7c-4f10-498b-b0e2-cc394e51a535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.958457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 782/1574] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:45:34.965770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8628d7c-4f10-498b-b0e2-cc394e51a535 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.965770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8628d7c-4f10-498b-b0e2-cc394e51a535 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.965770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8628d7c-4f10-498b-b0e2-cc394e51a535 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.966184 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba1162-4686-43fb-a7c3-e000c95412f7 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.966292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8628d7c-4f10-498b-b0e2-cc394e51a535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.966292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8628d7c-4f10-498b-b0e2-cc394e51a535 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.969204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba1162-4686-43fb-a7c3-e000c95412f7 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.973700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 793/1575] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:34.974023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817faf44-5f13-434c-bcd1-5b8f7a4f47a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.974511 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8628d7c-4f10-498b-b0e2-cc394e51a535 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:34.974641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817faf44-5f13-434c-bcd1-5b8f7a4f47a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.974884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817faf44-5f13-434c-bcd1-5b8f7a4f47a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.975153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817faf44-5f13-434c-bcd1-5b8f7a4f47a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.975778 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817faf44-5f13-434c-bcd1-5b8f7a4f47a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.977677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817faf44-5f13-434c-bcd1-5b8f7a4f47a0 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:34.983696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:34.985176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8628d7c-4f10-498b-b0e2-cc394e51a535 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:34.990122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ef864-262d-4be6-a9ad-8fb4fe4c448a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:34.990805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ef864-262d-4be6-a9ad-8fb4fe4c448a admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:34.991182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ef864-262d-4be6-a9ad-8fb4fe4c448a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:34.991477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ef864-262d-4be6-a9ad-8fb4fe4c448a admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:34.992109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ef864-262d-4be6-a9ad-8fb4fe4c448a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:34.994204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ef864-262d-4be6-a9ad-8fb4fe4c448a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.008278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 783/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:35.009086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817faf44-5f13-434c-bcd1-5b8f7a4f47a0 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.023997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 794/1577] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:34 2022] POST /identity/v3/users => generated 297 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:35.023997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ef864-262d-4be6-a9ad-8fb4fe4c448a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.027029 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:35.033415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ef864-262d-4be6-a9ad-8fb4fe4c448a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.071339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817faf44-5f13-434c-bcd1-5b8f7a4f47a0 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.076142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105bd0f-c435-46b8-9ddb-31080b21b921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.076746 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105bd0f-c435-46b8-9ddb-31080b21b921 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.077023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105bd0f-c435-46b8-9ddb-31080b21b921 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.077293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105bd0f-c435-46b8-9ddb-31080b21b921 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.078119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105bd0f-c435-46b8-9ddb-31080b21b921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.080370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105bd0f-c435-46b8-9ddb-31080b21b921 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.089604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e5097-47c0-4666-b480-45fc792ad1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.090253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e5097-47c0-4666-b480-45fc792ad1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.090528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e5097-47c0-4666-b480-45fc792ad1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.090811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e5097-47c0-4666-b480-45fc792ad1c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.091647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e5097-47c0-4666-b480-45fc792ad1c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.093857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e5097-47c0-4666-b480-45fc792ad1c9 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.101391 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 784/1578] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:35.102218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105bd0f-c435-46b8-9ddb-31080b21b921 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.112583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105bd0f-c435-46b8-9ddb-31080b21b921 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.118273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02157fc3-93ac-4fb7-b760-8534a7fadea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.118880 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02157fc3-93ac-4fb7-b760-8534a7fadea1 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.119789 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02157fc3-93ac-4fb7-b760-8534a7fadea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.120044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02157fc3-93ac-4fb7-b760-8534a7fadea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.120718 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02157fc3-93ac-4fb7-b760-8534a7fadea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.122896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02157fc3-93ac-4fb7-b760-8534a7fadea1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.127393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 795/1579] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:35.128017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e5097-47c0-4666-b480-45fc792ad1c9 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.153786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 785/1580] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:35.158211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02157fc3-93ac-4fb7-b760-8534a7fadea1 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.190026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e5097-47c0-4666-b480-45fc792ad1c9 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.218862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02157fc3-93ac-4fb7-b760-8534a7fadea1 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.224763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0c7bf-5fa4-47fe-840b-d3ad2f76c4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.225311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0c7bf-5fa4-47fe-840b-d3ad2f76c4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.225602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0c7bf-5fa4-47fe-840b-d3ad2f76c4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.226017 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0c7bf-5fa4-47fe-840b-d3ad2f76c4c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.226762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0c7bf-5fa4-47fe-840b-d3ad2f76c4c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.229195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0c7bf-5fa4-47fe-840b-d3ad2f76c4c4 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.245922 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48616e-d031-49a7-8104-06f2607f3d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.246692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48616e-d031-49a7-8104-06f2607f3d76 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.247006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48616e-d031-49a7-8104-06f2607f3d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.247316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48616e-d031-49a7-8104-06f2607f3d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.248090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48616e-d031-49a7-8104-06f2607f3d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.250342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48616e-d031-49a7-8104-06f2607f3d76 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.256044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 786/1581] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:35.256996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0c7bf-5fa4-47fe-840b-d3ad2f76c4c4 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.282205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 796/1582] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:35.282205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48616e-d031-49a7-8104-06f2607f3d76 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.300743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:35.303600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:35.324975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0c7bf-5fa4-47fe-840b-d3ad2f76c4c4 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.352240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48616e-d031-49a7-8104-06f2607f3d76 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.357529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066295c-8fab-47e6-a44d-6238c7181300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.358172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066295c-8fab-47e6-a44d-6238c7181300 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.358460 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066295c-8fab-47e6-a44d-6238c7181300 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.358736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066295c-8fab-47e6-a44d-6238c7181300 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.359378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066295c-8fab-47e6-a44d-6238c7181300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.361319 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066295c-8fab-47e6-a44d-6238c7181300 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.381569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5daa699-40d5-452a-9d55-cc7c4dcc4e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.382240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5daa699-40d5-452a-9d55-cc7c4dcc4e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.382518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5daa699-40d5-452a-9d55-cc7c4dcc4e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.382789 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5daa699-40d5-452a-9d55-cc7c4dcc4e32 admin admin] PATH_INFO: `/v3/projects/ed8dda1234f54905962e427bf75e82c1/users/ed4b7bfb87b144b38b82514c17a468bc/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.383012 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 797/1583] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:35.383827 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066295c-8fab-47e6-a44d-6238c7181300 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.395284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066295c-8fab-47e6-a44d-6238c7181300 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.400044 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb219e-678b-4ba7-9e9e-80f60c0e5729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.401135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5daa699-40d5-452a-9d55-cc7c4dcc4e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed8dda1234f54905962e427bf75e82c1, user_id=ed4b7bfb87b144b38b82514c17a468bc, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.401293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb219e-678b-4ba7-9e9e-80f60c0e5729 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.401470 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb219e-678b-4ba7-9e9e-80f60c0e5729 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.401754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb219e-678b-4ba7-9e9e-80f60c0e5729 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.402416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb219e-678b-4ba7-9e9e-80f60c0e5729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.403889 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5daa699-40d5-452a-9d55-cc7c4dcc4e32 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.404004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb219e-678b-4ba7-9e9e-80f60c0e5729 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.414930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 798/1584] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.415545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb219e-678b-4ba7-9e9e-80f60c0e5729 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.419255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 787/1585] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:35 2022] PUT /identity/v3/projects/ed8dda1234f54905962e427bf75e82c1/users/ed4b7bfb87b144b38b82514c17a468bc/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:35.419853 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5daa699-40d5-452a-9d55-cc7c4dcc4e32 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.425466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb219e-678b-4ba7-9e9e-80f60c0e5729 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.483964 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb11bf2-f3f2-4e97-ab73-30e72a2166e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.484178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5daa699-40d5-452a-9d55-cc7c4dcc4e32 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.484481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb11bf2-f3f2-4e97-ab73-30e72a2166e5 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.484758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb11bf2-f3f2-4e97-ab73-30e72a2166e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.485027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb11bf2-f3f2-4e97-ab73-30e72a2166e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.485663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb11bf2-f3f2-4e97-ab73-30e72a2166e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.487280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb11bf2-f3f2-4e97-ab73-30e72a2166e5 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.488993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a0bd4-b860-47f1-97a2-3bcfd00f126b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.489624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a0bd4-b860-47f1-97a2-3bcfd00f126b admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.489952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a0bd4-b860-47f1-97a2-3bcfd00f126b admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.490225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a0bd4-b860-47f1-97a2-3bcfd00f126b admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.490849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a0bd4-b860-47f1-97a2-3bcfd00f126b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.492839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a0bd4-b860-47f1-97a2-3bcfd00f126b admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.500009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 799/1586] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.500009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb11bf2-f3f2-4e97-ab73-30e72a2166e5 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.510474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb11bf2-f3f2-4e97-ab73-30e72a2166e5 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.516634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 788/1587] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:35.516634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a0bd4-b860-47f1-97a2-3bcfd00f126b admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.518492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17483234-f9e8-410d-83db-16e74a523f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.519086 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17483234-f9e8-410d-83db-16e74a523f8a admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.519413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17483234-f9e8-410d-83db-16e74a523f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.519686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17483234-f9e8-410d-83db-16e74a523f8a admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.520308 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17483234-f9e8-410d-83db-16e74a523f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.522286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17483234-f9e8-410d-83db-16e74a523f8a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.524236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a0bd4-b860-47f1-97a2-3bcfd00f126b admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.529384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6fec1-090f-4ba9-8581-67b939b960d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.530030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6fec1-090f-4ba9-8581-67b939b960d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.530339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6fec1-090f-4ba9-8581-67b939b960d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.530797 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad6fec1-090f-4ba9-8581-67b939b960d6 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.531430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6fec1-090f-4ba9-8581-67b939b960d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.534510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad6fec1-090f-4ba9-8581-67b939b960d6 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.544293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 800/1588] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/users => generated 303 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:35.545070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17483234-f9e8-410d-83db-16e74a523f8a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.555279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17483234-f9e8-410d-83db-16e74a523f8a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.561709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 789/1589] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:35.561709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6fec1-090f-4ba9-8581-67b939b960d6 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.561941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1875d-feed-446c-b4c6-ae5901e23e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.561941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1875d-feed-446c-b4c6-ae5901e23e57 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.561941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1875d-feed-446c-b4c6-ae5901e23e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.561941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1875d-feed-446c-b4c6-ae5901e23e57 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.562247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1875d-feed-446c-b4c6-ae5901e23e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.564195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1875d-feed-446c-b4c6-ae5901e23e57 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.569319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad6fec1-090f-4ba9-8581-67b939b960d6 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.575839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a00f8-ef4d-4a1e-902c-cb65b129afa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.576483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a00f8-ef4d-4a1e-902c-cb65b129afa3 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.576827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a00f8-ef4d-4a1e-902c-cb65b129afa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.577109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a00f8-ef4d-4a1e-902c-cb65b129afa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.577807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a00f8-ef4d-4a1e-902c-cb65b129afa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.579560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a00f8-ef4d-4a1e-902c-cb65b129afa3 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.589011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 801/1590] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:35.589011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1875d-feed-446c-b4c6-ae5901e23e57 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.590953 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 790/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.592161 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b50892-9157-4a0c-aa7f-a7b6bbf4d99d None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.592441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b50892-9157-4a0c-aa7f-a7b6bbf4d99d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.592715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b50892-9157-4a0c-aa7f-a7b6bbf4d99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.602096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1875d-feed-446c-b4c6-ae5901e23e57 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.607373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba8b0-b290-4263-bff1-f66b550ced34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.607980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba8b0-b290-4263-bff1-f66b550ced34 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.608262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba8b0-b290-4263-bff1-f66b550ced34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.608537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba8b0-b290-4263-bff1-f66b550ced34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.609179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba8b0-b290-4263-bff1-f66b550ced34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.610862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba8b0-b290-4263-bff1-f66b550ced34 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.612565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-45b50892-9157-4a0c-aa7f-a7b6bbf4d99d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:35.614237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-45b50892-9157-4a0c-aa7f-a7b6bbf4d99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:35.623174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 791/1592] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:35.623652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 802/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.623910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b50892-9157-4a0c-aa7f-a7b6bbf4d99d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.624594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba8b0-b290-4263-bff1-f66b550ced34 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.638958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b50892-9157-4a0c-aa7f-a7b6bbf4d99d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.639176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba8b0-b290-4263-bff1-f66b550ced34 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.643617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868714cc-6b0b-4ba3-a51b-81a0564bc35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.643927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6a16e-c8d5-4452-ba28-41de729f8d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.644278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868714cc-6b0b-4ba3-a51b-81a0564bc35c admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.644560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868714cc-6b0b-4ba3-a51b-81a0564bc35c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.644699 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6a16e-c8d5-4452-ba28-41de729f8d31 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.644868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868714cc-6b0b-4ba3-a51b-81a0564bc35c admin admin] PATH_INFO: `/v3/projects/56b8be344aab43f3a48f7b39c5dc9a03/users/38e2f5650c6a41e7ac723800c00ad77e/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.644991 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6a16e-c8d5-4452-ba28-41de729f8d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.645243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6a16e-c8d5-4452-ba28-41de729f8d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.645925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6a16e-c8d5-4452-ba28-41de729f8d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.647524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6a16e-c8d5-4452-ba28-41de729f8d31 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.658425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 803/1594] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.659372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6a16e-c8d5-4452-ba28-41de729f8d31 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.664709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868714cc-6b0b-4ba3-a51b-81a0564bc35c admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b8be344aab43f3a48f7b39c5dc9a03, user_id=38e2f5650c6a41e7ac723800c00ad77e, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.668016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868714cc-6b0b-4ba3-a51b-81a0564bc35c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.669623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6a16e-c8d5-4452-ba28-41de729f8d31 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.674693 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b30b98-8504-413a-889b-6eca5956fa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.675328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b30b98-8504-413a-889b-6eca5956fa3e admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.675591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b30b98-8504-413a-889b-6eca5956fa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.675987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b30b98-8504-413a-889b-6eca5956fa3e admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.676698 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b30b98-8504-413a-889b-6eca5956fa3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.677090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:35.678518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b30b98-8504-413a-889b-6eca5956fa3e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.682790 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 792/1595] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:35 2022] PUT /identity/v3/projects/56b8be344aab43f3a48f7b39c5dc9a03/users/38e2f5650c6a41e7ac723800c00ad77e/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:35.684527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868714cc-6b0b-4ba3-a51b-81a0564bc35c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.689676 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 804/1596] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.691178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b30b98-8504-413a-889b-6eca5956fa3e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.701710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b30b98-8504-413a-889b-6eca5956fa3e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.726391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:35.746846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868714cc-6b0b-4ba3-a51b-81a0564bc35c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.751684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6dc922-0d20-4cc4-b2fd-5d9af6a94bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.752296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6dc922-0d20-4cc4-b2fd-5d9af6a94bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.752578 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6dc922-0d20-4cc4-b2fd-5d9af6a94bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.752852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6dc922-0d20-4cc4-b2fd-5d9af6a94bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.753497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6dc922-0d20-4cc4-b2fd-5d9af6a94bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.755200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6dc922-0d20-4cc4-b2fd-5d9af6a94bc9 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.759507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109644f4-b6b4-495d-82fb-154c7f080a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.760629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109644f4-b6b4-495d-82fb-154c7f080a65 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.760910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109644f4-b6b4-495d-82fb-154c7f080a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.761183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109644f4-b6b4-495d-82fb-154c7f080a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.761842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109644f4-b6b4-495d-82fb-154c7f080a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.763421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109644f4-b6b4-495d-82fb-154c7f080a65 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.765715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 793/1597] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.766659 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6dc922-0d20-4cc4-b2fd-5d9af6a94bc9 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.776700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 805/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.777044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6dc922-0d20-4cc4-b2fd-5d9af6a94bc9 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.777337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109644f4-b6b4-495d-82fb-154c7f080a65 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.783548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f812759-a022-46de-88c8-e89c5fa0fef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.784137 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f812759-a022-46de-88c8-e89c5fa0fef9 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.784421 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f812759-a022-46de-88c8-e89c5fa0fef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.784698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f812759-a022-46de-88c8-e89c5fa0fef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.785326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f812759-a022-46de-88c8-e89c5fa0fef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.786622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109644f4-b6b4-495d-82fb-154c7f080a65 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.786936 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f812759-a022-46de-88c8-e89c5fa0fef9 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.791121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857822f-c4bb-49c0-935b-606724ceff54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.791740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857822f-c4bb-49c0-935b-606724ceff54 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.792371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857822f-c4bb-49c0-935b-606724ceff54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.792371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857822f-c4bb-49c0-935b-606724ceff54 admin admin] PATH_INFO: `/v3/projects/0b7703aae5634dabb496bc221882863c/users/acf57eb971eb4f78b173a5ab47160bd1/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.797157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 794/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.798056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f812759-a022-46de-88c8-e89c5fa0fef9 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.806702 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857822f-c4bb-49c0-935b-606724ceff54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7703aae5634dabb496bc221882863c, user_id=acf57eb971eb4f78b173a5ab47160bd1, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.809346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857822f-c4bb-49c0-935b-606724ceff54 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.824229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 806/1600] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:35 2022] PUT /identity/v3/projects/0b7703aae5634dabb496bc221882863c/users/acf57eb971eb4f78b173a5ab47160bd1/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:35.825109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857822f-c4bb-49c0-935b-606724ceff54 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.857186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f812759-a022-46de-88c8-e89c5fa0fef9 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.886084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857822f-c4bb-49c0-935b-606724ceff54 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.890867 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe81b64-0221-48c8-8c8d-943c0ee738a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.891465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe81b64-0221-48c8-8c8d-943c0ee738a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.891743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe81b64-0221-48c8-8c8d-943c0ee738a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.892014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe81b64-0221-48c8-8c8d-943c0ee738a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.892648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe81b64-0221-48c8-8c8d-943c0ee738a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.894277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe81b64-0221-48c8-8c8d-943c0ee738a3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.904825 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 807/1601] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.905444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe81b64-0221-48c8-8c8d-943c0ee738a3 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.906848 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25833e94-58f9-4074-96f8-74a4f1d55d3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.907626 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25833e94-58f9-4074-96f8-74a4f1d55d3d service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.908286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25833e94-58f9-4074-96f8-74a4f1d55d3d service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.908286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25833e94-58f9-4074-96f8-74a4f1d55d3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.914818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe81b64-0221-48c8-8c8d-943c0ee738a3 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.917986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25833e94-58f9-4074-96f8-74a4f1d55d3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.919357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12c039-6df9-48fb-b431-3c501f48d471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.919649 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25833e94-58f9-4074-96f8-74a4f1d55d3d service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.919934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12c039-6df9-48fb-b431-3c501f48d471 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.920210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12c039-6df9-48fb-b431-3c501f48d471 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.920480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12c039-6df9-48fb-b431-3c501f48d471 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.921100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12c039-6df9-48fb-b431-3c501f48d471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.922703 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12c039-6df9-48fb-b431-3c501f48d471 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.929242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 795/1602] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:35.930072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25833e94-58f9-4074-96f8-74a4f1d55d3d service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.936122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 808/1603] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.936122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12c039-6df9-48fb-b431-3c501f48d471 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.939264 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25833e94-58f9-4074-96f8-74a4f1d55d3d service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.945208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767429e9-675b-4538-996c-fbb19f5bfba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.945989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767429e9-675b-4538-996c-fbb19f5bfba6 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.946284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12c039-6df9-48fb-b431-3c501f48d471 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.946420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767429e9-675b-4538-996c-fbb19f5bfba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.946736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767429e9-675b-4538-996c-fbb19f5bfba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.948208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767429e9-675b-4538-996c-fbb19f5bfba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.949943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767429e9-675b-4538-996c-fbb19f5bfba6 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.951681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1324111-9950-4636-b168-5c4206eabb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.952266 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1324111-9950-4636-b168-5c4206eabb16 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.952615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1324111-9950-4636-b168-5c4206eabb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.952983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1324111-9950-4636-b168-5c4206eabb16 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.953497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1324111-9950-4636-b168-5c4206eabb16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.955332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1324111-9950-4636-b168-5c4206eabb16 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.961109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 796/1604] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.962021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767429e9-675b-4538-996c-fbb19f5bfba6 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.972051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 809/1605] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.972461 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767429e9-675b-4538-996c-fbb19f5bfba6 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.973435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1324111-9950-4636-b168-5c4206eabb16 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:35.977189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf32d1-6a0a-4747-8478-62e82a79793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.977817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf32d1-6a0a-4747-8478-62e82a79793e admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.978109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf32d1-6a0a-4747-8478-62e82a79793e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.978405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf32d1-6a0a-4747-8478-62e82a79793e admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.979046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf32d1-6a0a-4747-8478-62e82a79793e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:35.980651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf32d1-6a0a-4747-8478-62e82a79793e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:35.984556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1324111-9950-4636-b168-5c4206eabb16 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:35.989680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61afb4-a645-4e4d-9996-f3a1e7038a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:35.990394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61afb4-a645-4e4d-9996-f3a1e7038a89 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.990679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61afb4-a645-4e4d-9996-f3a1e7038a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.990938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 797/1606] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:35.991296 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61afb4-a645-4e4d-9996-f3a1e7038a89 admin admin] PATH_INFO: `/v3/projects/9bd782f79ed84b81a998ae437af2a6cb/users/3c0b7528b0ff4b09923a0e951d052ed5/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:35.995988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922c11a-3780-443b-961d-bcee732cd5e8 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:35.996514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922c11a-3780-443b-961d-bcee732cd5e8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:35.996912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922c11a-3780-443b-961d-bcee732cd5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.007194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61afb4-a645-4e4d-9996-f3a1e7038a89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd782f79ed84b81a998ae437af2a6cb, user_id=3c0b7528b0ff4b09923a0e951d052ed5, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:36.010467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61afb4-a645-4e4d-9996-f3a1e7038a89 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:36.020573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-e922c11a-3780-443b-961d-bcee732cd5e8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:36.021855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e922c11a-3780-443b-961d-bcee732cd5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:36.028564 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 810/1607] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:35 2022] PUT /identity/v3/projects/9bd782f79ed84b81a998ae437af2a6cb/users/3c0b7528b0ff4b09923a0e951d052ed5/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:36.030009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61afb4-a645-4e4d-9996-f3a1e7038a89 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:36.045774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:36.085475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:36.099352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 798/1608] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:36.101348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61afb4-a645-4e4d-9996-f3a1e7038a89 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:36.102910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922c11a-3780-443b-961d-bcee732cd5e8 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:36.108646 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7840c76-7bab-4cf1-83ef-c7332f84a422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:36.110216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7840c76-7bab-4cf1-83ef-c7332f84a422 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.110456 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7840c76-7bab-4cf1-83ef-c7332f84a422 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.110779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7840c76-7bab-4cf1-83ef-c7332f84a422 admin admin] PATH_INFO: `/v3/projects/7229cfcf52b64e83914f4ea2eb5627dd/users/980f097960df41528ac45d6c5a07c5f5/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.121196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922c11a-3780-443b-961d-bcee732cd5e8 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:36.129782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17ca59-5f47-4c0e-8be2-ac8fd8b5598e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:36.130637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17ca59-5f47-4c0e-8be2-ac8fd8b5598e admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.130926 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17ca59-5f47-4c0e-8be2-ac8fd8b5598e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.131199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17ca59-5f47-4c0e-8be2-ac8fd8b5598e admin admin] PATH_INFO: `/v3/projects/56b8be344aab43f3a48f7b39c5dc9a03/users/38e2f5650c6a41e7ac723800c00ad77e/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.133143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17ca59-5f47-4c0e-8be2-ac8fd8b5598e admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b8be344aab43f3a48f7b39c5dc9a03, user_id=38e2f5650c6a41e7ac723800c00ad77e, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:36.135881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7840c76-7bab-4cf1-83ef-c7332f84a422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7229cfcf52b64e83914f4ea2eb5627dd, user_id=980f097960df41528ac45d6c5a07c5f5, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:36.136451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17ca59-5f47-4c0e-8be2-ac8fd8b5598e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:36.139130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7840c76-7bab-4cf1-83ef-c7332f84a422 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:36.149440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 799/1609] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:36 2022] PUT /identity/v3/projects/56b8be344aab43f3a48f7b39c5dc9a03/users/38e2f5650c6a41e7ac723800c00ad77e/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:36.150647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17ca59-5f47-4c0e-8be2-ac8fd8b5598e admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:36.161493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 811/1610] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:36 2022] PUT /identity/v3/projects/7229cfcf52b64e83914f4ea2eb5627dd/users/980f097960df41528ac45d6c5a07c5f5/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:36.162271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7840c76-7bab-4cf1-83ef-c7332f84a422 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:36.229050 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7840c76-7bab-4cf1-83ef-c7332f84a422 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:36.230892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17ca59-5f47-4c0e-8be2-ac8fd8b5598e admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:36.234138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c8ee4-7039-42f3-99e1-b32cac4cf467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:36.234827 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c8ee4-7039-42f3-99e1-b32cac4cf467 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.235109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c8ee4-7039-42f3-99e1-b32cac4cf467 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.235381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c8ee4-7039-42f3-99e1-b32cac4cf467 admin admin] PATH_INFO: `/v3/projects/832db97796824ce793ef6afc423c1c12/users/2f84f6f1ccf74df9aa7105448a7b774b/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.235976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbe66c-5cbb-438b-8475-e3fc9bc0c8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:36.236739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbe66c-5cbb-438b-8475-e3fc9bc0c8ab admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.237099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbe66c-5cbb-438b-8475-e3fc9bc0c8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.237633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbe66c-5cbb-438b-8475-e3fc9bc0c8ab admin admin] PATH_INFO: `/v3/projects/60c4101d507d4b618cf45d535eb6170c/users/8532c435f485429bb5b9acc29236cfbb/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.255751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c8ee4-7039-42f3-99e1-b32cac4cf467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=832db97796824ce793ef6afc423c1c12, user_id=2f84f6f1ccf74df9aa7105448a7b774b, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:36.258542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbe66c-5cbb-438b-8475-e3fc9bc0c8ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c4101d507d4b618cf45d535eb6170c, user_id=8532c435f485429bb5b9acc29236cfbb, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:36.260352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c8ee4-7039-42f3-99e1-b32cac4cf467 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:36.262362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbe66c-5cbb-438b-8475-e3fc9bc0c8ab admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:36.277202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 812/1611] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:36 2022] PUT /identity/v3/projects/832db97796824ce793ef6afc423c1c12/users/2f84f6f1ccf74df9aa7105448a7b774b/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:36.288924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 800/1612] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:36 2022] PUT /identity/v3/projects/60c4101d507d4b618cf45d535eb6170c/users/8532c435f485429bb5b9acc29236cfbb/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:36.288924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbe66c-5cbb-438b-8475-e3fc9bc0c8ab admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:36.290970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fabb2-c326-4281-a134-1e9f0bda7f07 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.291498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fabb2-c326-4281-a134-1e9f0bda7f07 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.291830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fabb2-c326-4281-a134-1e9f0bda7f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.312982 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-415fabb2-c326-4281-a134-1e9f0bda7f07 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:36.314140 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-415fabb2-c326-4281-a134-1e9f0bda7f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:36.361258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbe66c-5cbb-438b-8475-e3fc9bc0c8ab admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:36.367441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bbd4e-898d-494f-99ee-a031763534b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:36.368203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bbd4e-898d-494f-99ee-a031763534b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.368549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bbd4e-898d-494f-99ee-a031763534b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.368891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bbd4e-898d-494f-99ee-a031763534b5 admin admin] PATH_INFO: `/v3/projects/281ccf5c1a784a89b9f4f5e0905f33ca/users/973b2f1bc42a48a7bc488889d5565944/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.384226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 813/1613] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:36.386168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fabb2-c326-4281-a134-1e9f0bda7f07 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:36.389105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bbd4e-898d-494f-99ee-a031763534b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281ccf5c1a784a89b9f4f5e0905f33ca, user_id=973b2f1bc42a48a7bc488889d5565944, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:36.392939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bbd4e-898d-494f-99ee-a031763534b5 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:36.394511 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-415fabb2-c326-4281-a134-1e9f0bda7f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:36.409902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 801/1614] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:36 2022] PUT /identity/v3/projects/281ccf5c1a784a89b9f4f5e0905f33ca/users/973b2f1bc42a48a7bc488889d5565944/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:36.411789 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a79a7-6790-45f5-affe-a362e9978755 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.411915 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a79a7-6790-45f5-affe-a362e9978755 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.412383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a79a7-6790-45f5-affe-a362e9978755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.445450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-1c6a79a7-6790-45f5-affe-a362e9978755 None None] MFA Rules not processed for user `38e2f5650c6a41e7ac723800c00ad77e`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:36.446945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1c6a79a7-6790-45f5-affe-a362e9978755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:36.452140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:36.476224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fabb2-c326-4281-a134-1e9f0bda7f07 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:36.485730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:36.529460 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 802/1615] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:36.531154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfbb10-3d79-4803-a6a0-96551b487e08 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.531444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfbb10-3d79-4803-a6a0-96551b487e08 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.531850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfbb10-3d79-4803-a6a0-96551b487e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.535075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5379c6b-32d9-4d57-bdea-16279f23d9b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:36.535496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5379c6b-32d9-4d57-bdea-16279f23d9b9 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.535872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5379c6b-32d9-4d57-bdea-16279f23d9b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.536240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5379c6b-32d9-4d57-bdea-16279f23d9b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.556348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-fdbfbb10-3d79-4803-a6a0-96551b487e08 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:36.558427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-fdbfbb10-3d79-4803-a6a0-96551b487e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:36.608739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5379c6b-32d9-4d57-bdea-16279f23d9b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:36.610314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5379c6b-32d9-4d57-bdea-16279f23d9b9 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:36.622996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 814/1616] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:36.624321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de70b3-b8cb-4a8f-8900-ae6d0e1abb6f None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.624621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de70b3-b8cb-4a8f-8900-ae6d0e1abb6f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.624944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de70b3-b8cb-4a8f-8900-ae6d0e1abb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.636139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 803/1617] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:36.636139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a34c9e-fde4-45c8-bbd8-374300f5c2b8 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.636139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a34c9e-fde4-45c8-bbd8-374300f5c2b8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.636139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a34c9e-fde4-45c8-bbd8-374300f5c2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.666861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-64a34c9e-fde4-45c8-bbd8-374300f5c2b8 None None] MFA Rules not processed for user `2f84f6f1ccf74df9aa7105448a7b774b`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:36.668391 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-64a34c9e-fde4-45c8-bbd8-374300f5c2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:36.669005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-92de70b3-b8cb-4a8f-8900-ae6d0e1abb6f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:36.670268 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-92de70b3-b8cb-4a8f-8900-ae6d0e1abb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:36.680133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 815/1618] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:36.680982 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de70b3-b8cb-4a8f-8900-ae6d0e1abb6f None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:36.693085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de70b3-b8cb-4a8f-8900-ae6d0e1abb6f None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:36.697505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f70a30-4b10-46dd-830f-092aadf81d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:36.698199 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f70a30-4b10-46dd-830f-092aadf81d6e service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.699004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f70a30-4b10-46dd-830f-092aadf81d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.699357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f70a30-4b10-46dd-830f-092aadf81d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.711239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f70a30-4b10-46dd-830f-092aadf81d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:36.713299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f70a30-4b10-46dd-830f-092aadf81d6e service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:36.725587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 816/1619] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:36.726926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afe7cf-191e-4fda-9c56-64ff4ddf3128 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.728045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afe7cf-191e-4fda-9c56-64ff4ddf3128 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.728045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afe7cf-191e-4fda-9c56-64ff4ddf3128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.761022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d0afe7cf-191e-4fda-9c56-64ff4ddf3128 None None] MFA Rules not processed for user `973b2f1bc42a48a7bc488889d5565944`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:36.764160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d0afe7cf-191e-4fda-9c56-64ff4ddf3128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:36.801352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 804/1620] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:36.804417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a34c9e-fde4-45c8-bbd8-374300f5c2b8 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:36.886885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 817/1621] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:36.888088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afe7cf-191e-4fda-9c56-64ff4ddf3128 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:36.911259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:36.927080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afe7cf-191e-4fda-9c56-64ff4ddf3128 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:36.940589 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba8dbe-9c66-4ce2-955e-d5915925cc12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:36.942112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba8dbe-9c66-4ce2-955e-d5915925cc12 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.945312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba8dbe-9c66-4ce2-955e-d5915925cc12 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.946068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba8dbe-9c66-4ce2-955e-d5915925cc12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.969415 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a34c9e-fde4-45c8-bbd8-374300f5c2b8 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:36.974004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba8dbe-9c66-4ce2-955e-d5915925cc12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:36.975515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7610c-52fc-41e8-998b-a0fafefd6bef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:36.976138 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7610c-52fc-41e8-998b-a0fafefd6bef service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:36.976419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7610c-52fc-41e8-998b-a0fafefd6bef service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:36.976693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7610c-52fc-41e8-998b-a0fafefd6bef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:36.977511 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba8dbe-9c66-4ce2-955e-d5915925cc12 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:36.990271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 818/1622] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:36.991479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba8dbe-9c66-4ce2-955e-d5915925cc12 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:37.000667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba8dbe-9c66-4ce2-955e-d5915925cc12 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:37.005172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f25818-f271-410c-8217-d50a02ca0333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:37.006139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f25818-f271-410c-8217-d50a02ca0333 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:37.006647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f25818-f271-410c-8217-d50a02ca0333 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:37.007158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f25818-f271-410c-8217-d50a02ca0333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:37.018797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f25818-f271-410c-8217-d50a02ca0333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:37.021405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f25818-f271-410c-8217-d50a02ca0333 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:37.037071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 819/1623] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:37.039158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5339be-1e94-4af6-9b66-07afc42eee59 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:37.039858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5339be-1e94-4af6-9b66-07afc42eee59 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:37.040387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5339be-1e94-4af6-9b66-07afc42eee59 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:37.043087 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 820/1624] 10.222.0.26 () {54 vars in 961 bytes} [Tue Oct 4 08:45:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:37.044794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61200954-69ad-4f4b-a40c-a0257df686b0 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:37.045532 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61200954-69ad-4f4b-a40c-a0257df686b0 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:37.046065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61200954-69ad-4f4b-a40c-a0257df686b0 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:37.047982 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 821/1625] 10.222.0.26 () {54 vars in 961 bytes} [Tue Oct 4 08:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:37.049404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f71f3-8275-4dcb-812d-44662dd28576 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:37.050506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f71f3-8275-4dcb-812d-44662dd28576 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:37.051176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f71f3-8275-4dcb-812d-44662dd28576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:37.066707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7610c-52fc-41e8-998b-a0fafefd6bef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:37.068559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7610c-52fc-41e8-998b-a0fafefd6bef service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:37.081410 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:37.088251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 805/1626] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:37.089398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08875a-df5d-4503-8c28-4eff00079fd6 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:37.089772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08875a-df5d-4503-8c28-4eff00079fd6 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:37.090016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08875a-df5d-4503-8c28-4eff00079fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:37.146330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-7c08875a-df5d-4503-8c28-4eff00079fd6 None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:37.147556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7c08875a-df5d-4503-8c28-4eff00079fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:37.164389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-f86f71f3-8275-4dcb-812d-44662dd28576 None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:37.168562 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f86f71f3-8275-4dcb-812d-44662dd28576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:37.245729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 806/1627] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Oct 4 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:37.263428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08875a-df5d-4503-8c28-4eff00079fd6 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:37.289358 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08875a-df5d-4503-8c28-4eff00079fd6 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:37.306256 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458fc2b-f231-4798-991c-e26949aa8a5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:37.307264 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458fc2b-f231-4798-991c-e26949aa8a5b service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:37.307768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458fc2b-f231-4798-991c-e26949aa8a5b service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:37.308692 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458fc2b-f231-4798-991c-e26949aa8a5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:37.318989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458fc2b-f231-4798-991c-e26949aa8a5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:37.321274 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458fc2b-f231-4798-991c-e26949aa8a5b service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:37.330983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 822/1628] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Oct 4 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:37.333834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 807/1629] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:37.344744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f71f3-8275-4dcb-812d-44662dd28576 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:37.372557 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f71f3-8275-4dcb-812d-44662dd28576 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:37.390878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fdb185-ad6c-4c04-8b77-f38094df1f7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:37.391703 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdb185-ad6c-4c04-8b77-f38094df1f7b service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:37.392083 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdb185-ad6c-4c04-8b77-f38094df1f7b service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:37.392459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdb185-ad6c-4c04-8b77-f38094df1f7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:37.415548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fdb185-ad6c-4c04-8b77-f38094df1f7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:37.417607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fdb185-ad6c-4c04-8b77-f38094df1f7b service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:37.438408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 823/1630] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:37.646415 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83848d-5cf3-4953-9a47-fd1e5fc34e6a None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:37.646724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83848d-5cf3-4953-9a47-fd1e5fc34e6a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:37.647026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83848d-5cf3-4953-9a47-fd1e5fc34e6a None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:37.649741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 808/1631] 10.222.0.26 () {54 vars in 912 bytes} [Tue Oct 4 08:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:37.656999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89619120-6c99-4e0c-abe5-780d0c1367c7 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:37.657380 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89619120-6c99-4e0c-abe5-780d0c1367c7 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:37.661283 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89619120-6c99-4e0c-abe5-780d0c1367c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:37.696263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-89619120-6c99-4e0c-abe5-780d0c1367c7 None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:37.699337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-89619120-6c99-4e0c-abe5-780d0c1367c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:37.727696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 824/1632] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:45:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:37.737104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83848d-5cf3-4953-9a47-fd1e5fc34e6a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:37.755409 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83848d-5cf3-4953-9a47-fd1e5fc34e6a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:37.764139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12718163-a2f4-4f40-982f-15104600c175 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:37.767016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12718163-a2f4-4f40-982f-15104600c175 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:37.767238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12718163-a2f4-4f40-982f-15104600c175 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:37.768372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12718163-a2f4-4f40-982f-15104600c175 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:37.799163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12718163-a2f4-4f40-982f-15104600c175 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:37.803852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12718163-a2f4-4f40-982f-15104600c175 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:37.830918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 809/1633] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:38.045486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318fcbf-7add-4314-b385-dca2da75b26a None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:38.046539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318fcbf-7add-4314-b385-dca2da75b26a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:38.047271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318fcbf-7add-4314-b385-dca2da75b26a None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:38.049187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 825/1634] 10.222.0.26 () {54 vars in 912 bytes} [Tue Oct 4 08:45:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:38.055747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71c77c-ec3f-4130-9f9c-0462346e08f1 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:38.056278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71c77c-ec3f-4130-9f9c-0462346e08f1 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:38.056414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71c77c-ec3f-4130-9f9c-0462346e08f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:38.136995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-2c71c77c-ec3f-4130-9f9c-0462346e08f1 None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:38.143024 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2c71c77c-ec3f-4130-9f9c-0462346e08f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:38.172217 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 810/1635] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:45:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:45:38.260079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318fcbf-7add-4314-b385-dca2da75b26a None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:38.272969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318fcbf-7add-4314-b385-dca2da75b26a None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:38.281052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e226cd-3787-4ff8-acde-652192d34984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:38.281999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e226cd-3787-4ff8-acde-652192d34984 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:38.289842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e226cd-3787-4ff8-acde-652192d34984 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:38.290411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e226cd-3787-4ff8-acde-652192d34984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:38.329074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e226cd-3787-4ff8-acde-652192d34984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:38.331604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e226cd-3787-4ff8-acde-652192d34984 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:38.361532 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 826/1636] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:38.609510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71c77c-ec3f-4130-9f9c-0462346e08f1 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:38.629627 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71c77c-ec3f-4130-9f9c-0462346e08f1 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:38.645119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:38.646010 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:38.646888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:38.646888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] PATH_INFO: `/v3/users/2f84f6f1ccf74df9aa7105448a7b774b` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:38.648546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f84f6f1ccf74df9aa7105448a7b774b)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:38.651522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:38.701682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:38.731893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f84f6f1ccf74df9aa7105448a7b774b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:38.734193 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f84f6f1ccf74df9aa7105448a7b774b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:38.773428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f84f6f1ccf74df9aa7105448a7b774b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:38.797131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f84f6f1ccf74df9aa7105448a7b774b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:38.810681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f84f6f1ccf74df9aa7105448a7b774b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:38.856178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 811/1637] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:45:38 2022] DELETE /identity/v3/users/2f84f6f1ccf74df9aa7105448a7b774b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:38.860938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f1415-51f3-4cae-ba6c-a01b1138283a None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:38.861225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f1415-51f3-4cae-ba6c-a01b1138283a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:38.861959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f1415-51f3-4cae-ba6c-a01b1138283a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:38.908625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-9b7f1415-51f3-4cae-ba6c-a01b1138283a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:38.912316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9b7f1415-51f3-4cae-ba6c-a01b1138283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:39.008179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 827/1638] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:39.046881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:39.048356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:39.223129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a179e8a-c569-4c54-8b36-4b07b4c77395 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:39.368661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa119b-cff8-4011-9cee-2751d7bf7185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:39.368661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa119b-cff8-4011-9cee-2751d7bf7185 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:39.368661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa119b-cff8-4011-9cee-2751d7bf7185 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:39.368661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa119b-cff8-4011-9cee-2751d7bf7185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:39.370146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa119b-cff8-4011-9cee-2751d7bf7185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:39.370146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa119b-cff8-4011-9cee-2751d7bf7185 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:39.370146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 812/1639] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:40.084099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f5ac0-bafb-4ac4-8e34-1f74fd337b08 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:40.085252 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f5ac0-bafb-4ac4-8e34-1f74fd337b08 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:40.085709 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f5ac0-bafb-4ac4-8e34-1f74fd337b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:40.155666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6f5f5ac0-bafb-4ac4-8e34-1f74fd337b08 None None] MFA Rules not processed for user `8532c435f485429bb5b9acc29236cfbb`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:40.158710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6f5f5ac0-bafb-4ac4-8e34-1f74fd337b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:40.269226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:40.329038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 828/1640] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:40 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:40.335175 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa119b-cff8-4011-9cee-2751d7bf7185 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:40.339254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-13aa119b-cff8-4011-9cee-2751d7bf7185 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:40.365482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa119b-cff8-4011-9cee-2751d7bf7185 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:40.390327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6c964-245d-4f57-ab5b-54fde46d8b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:40.391188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6c964-245d-4f57-ab5b-54fde46d8b8c admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:40.392320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6c964-245d-4f57-ab5b-54fde46d8b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:40.393776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b6c964-245d-4f57-ab5b-54fde46d8b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:40.397950 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6c964-245d-4f57-ab5b-54fde46d8b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:40.400849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b6c964-245d-4f57-ab5b-54fde46d8b8c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:40.489239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 813/1641] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:40 2022] POST /identity/v3/projects => generated 405 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:40.490614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f5ac0-bafb-4ac4-8e34-1f74fd337b08 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:40.509009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f5ac0-bafb-4ac4-8e34-1f74fd337b08 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:40.637343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4858af1-f4fa-4b1a-a669-30ca84758a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:40.638011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4858af1-f4fa-4b1a-a669-30ca84758a40 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:40.638297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4858af1-f4fa-4b1a-a669-30ca84758a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:40.638575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4858af1-f4fa-4b1a-a669-30ca84758a40 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:40.639240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4858af1-f4fa-4b1a-a669-30ca84758a40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:40.641209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4858af1-f4fa-4b1a-a669-30ca84758a40 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:40.698835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 829/1642] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:40 2022] POST /identity/v3/users => generated 309 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:40.702669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6c964-245d-4f57-ab5b-54fde46d8b8c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:40.746079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b6c964-245d-4f57-ab5b-54fde46d8b8c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:40.769136 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546eff4d-8dd9-42a9-be8d-b5efe73dba21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:40.769488 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546eff4d-8dd9-42a9-be8d-b5efe73dba21 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:40.769801 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546eff4d-8dd9-42a9-be8d-b5efe73dba21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:40.770078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546eff4d-8dd9-42a9-be8d-b5efe73dba21 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:40.770733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546eff4d-8dd9-42a9-be8d-b5efe73dba21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:40.772749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546eff4d-8dd9-42a9-be8d-b5efe73dba21 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:40.815017 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 814/1643] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:40.815468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4858af1-f4fa-4b1a-a669-30ca84758a40 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:40.835853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4858af1-f4fa-4b1a-a669-30ca84758a40 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:40.845573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87facb7-511a-4877-97dc-6dcaeb0f849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:40.847411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87facb7-511a-4877-97dc-6dcaeb0f849e admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:40.848133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87facb7-511a-4877-97dc-6dcaeb0f849e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:40.848555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87facb7-511a-4877-97dc-6dcaeb0f849e admin admin] PATH_INFO: `/v3/projects/24cece7ccdb746669c41342bd7f45305/users/3c5c5182fa3e46cab4759d2782feb98e/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:40.879002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87facb7-511a-4877-97dc-6dcaeb0f849e admin admin] RBAC: Authorizing `identity:create_grant(project_id=24cece7ccdb746669c41342bd7f45305, user_id=3c5c5182fa3e46cab4759d2782feb98e, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:40.883846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87facb7-511a-4877-97dc-6dcaeb0f849e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:40.912267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 830/1644] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:40 2022] PUT /identity/v3/projects/24cece7ccdb746669c41342bd7f45305/users/3c5c5182fa3e46cab4759d2782feb98e/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:45:40.917036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546eff4d-8dd9-42a9-be8d-b5efe73dba21 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:40.929270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546eff4d-8dd9-42a9-be8d-b5efe73dba21 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:41.018504 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a1556-d6e2-4b51-97a0-cc9a565cc992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:41.019261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a1556-d6e2-4b51-97a0-cc9a565cc992 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:41.019607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a1556-d6e2-4b51-97a0-cc9a565cc992 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:41.020223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a1556-d6e2-4b51-97a0-cc9a565cc992 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:41.021278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a1556-d6e2-4b51-97a0-cc9a565cc992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:41.024260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a1556-d6e2-4b51-97a0-cc9a565cc992 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:41.040567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 815/1645] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:41.042832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87facb7-511a-4877-97dc-6dcaeb0f849e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:41.061323 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87facb7-511a-4877-97dc-6dcaeb0f849e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:41.106446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a32ae2-f695-4aa2-9fff-7801e72659ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:41.108337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a32ae2-f695-4aa2-9fff-7801e72659ec admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:41.109584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a32ae2-f695-4aa2-9fff-7801e72659ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:41.109896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a32ae2-f695-4aa2-9fff-7801e72659ec admin admin] PATH_INFO: `/v3/projects/24cece7ccdb746669c41342bd7f45305/users/3c5c5182fa3e46cab4759d2782feb98e/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:41.118454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a32ae2-f695-4aa2-9fff-7801e72659ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=24cece7ccdb746669c41342bd7f45305, user_id=3c5c5182fa3e46cab4759d2782feb98e, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:41.128629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a32ae2-f695-4aa2-9fff-7801e72659ec admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:41.172958 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 831/1646] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:41 2022] PUT /identity/v3/projects/24cece7ccdb746669c41342bd7f45305/users/3c5c5182fa3e46cab4759d2782feb98e/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:41.481115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a1556-d6e2-4b51-97a0-cc9a565cc992 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:41.484625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b80a1556-d6e2-4b51-97a0-cc9a565cc992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:41.518476 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a1556-d6e2-4b51-97a0-cc9a565cc992 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:41.558056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:41.690123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:41.691157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:41.691619 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:41.692002 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] PATH_INFO: `/v3/projects/832db97796824ce793ef6afc423c1c12` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:41.694267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=832db97796824ce793ef6afc423c1c12)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:41.695839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: WARNING py.warnings [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:45:41.695839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: warnings.warn(msg) Oct 04 08:45:41.695839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: Oct 04 08:45:41.699070 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:41.819076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '832db97796824ce793ef6afc423c1c12', 'request_id': 'req-f280e231-de83-4e74-9cd5-b9272628db47'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:41.861351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '832db97796824ce793ef6afc423c1c12', 'request_id': 'req-f280e231-de83-4e74-9cd5-b9272628db47'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:41.887731 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '832db97796824ce793ef6afc423c1c12', 'request_id': 'req-f280e231-de83-4e74-9cd5-b9272628db47'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:41.922559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] The token cache is being invalidate because project 832db97796824ce793ef6afc423c1c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:45:41.922918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:41.926131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f280e231-de83-4e74-9cd5-b9272628db47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:41.932113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 816/1647] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:45:41 2022] DELETE /identity/v3/projects/832db97796824ce793ef6afc423c1c12 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:41.934138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ddb53-919d-40a1-a0d2-56ef4cb7b206 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:41.934545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ddb53-919d-40a1-a0d2-56ef4cb7b206 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:41.935262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ddb53-919d-40a1-a0d2-56ef4cb7b206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:42.015445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-991ddb53-919d-40a1-a0d2-56ef4cb7b206 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:42.017631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-991ddb53-919d-40a1-a0d2-56ef4cb7b206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:42.232109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 832/1648] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:42.239875 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f55f5-8682-4fc7-9c8f-821d0b9f2527 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:42.240267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f55f5-8682-4fc7-9c8f-821d0b9f2527 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:42.240615 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f55f5-8682-4fc7-9c8f-821d0b9f2527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:42.288905 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d72f55f5-8682-4fc7-9c8f-821d0b9f2527 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:42.290231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d72f55f5-8682-4fc7-9c8f-821d0b9f2527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:42.318251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 817/1649] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:42.323261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ddb53-919d-40a1-a0d2-56ef4cb7b206 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:42.344288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ddb53-919d-40a1-a0d2-56ef4cb7b206 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:42.351077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dfaa8-4d59-42c6-82f6-8696d733228b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:42.351971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dfaa8-4d59-42c6-82f6-8696d733228b admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:42.352323 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dfaa8-4d59-42c6-82f6-8696d733228b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:42.352668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5dfaa8-4d59-42c6-82f6-8696d733228b admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:42.354472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dfaa8-4d59-42c6-82f6-8696d733228b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:45:42.354853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dfaa8-4d59-42c6-82f6-8696d733228b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:42.356617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5dfaa8-4d59-42c6-82f6-8696d733228b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:42.384706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 833/1650] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:45:42.385436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f55f5-8682-4fc7-9c8f-821d0b9f2527 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:42.399009 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f55f5-8682-4fc7-9c8f-821d0b9f2527 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:42.406131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e066f-0af5-4b7b-9bbb-a0b36a9c7195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:42.406801 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e066f-0af5-4b7b-9bbb-a0b36a9c7195 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:42.407057 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e066f-0af5-4b7b-9bbb-a0b36a9c7195 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:42.407338 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e066f-0af5-4b7b-9bbb-a0b36a9c7195 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:42.407981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e066f-0af5-4b7b-9bbb-a0b36a9c7195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:42.411207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e066f-0af5-4b7b-9bbb-a0b36a9c7195 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:42.465000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 818/1651] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:42 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:42.467458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dfaa8-4d59-42c6-82f6-8696d733228b admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:42.485338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5dfaa8-4d59-42c6-82f6-8696d733228b admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:42.547149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:42.676121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b96d5b-a7e1-419b-870d-7c0a9edc4730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:42.676764 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b96d5b-a7e1-419b-870d-7c0a9edc4730 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:42.677052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b96d5b-a7e1-419b-870d-7c0a9edc4730 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:42.677333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b96d5b-a7e1-419b-870d-7c0a9edc4730 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:42.678035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b96d5b-a7e1-419b-870d-7c0a9edc4730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:42.680026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b96d5b-a7e1-419b-870d-7c0a9edc4730 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:42.729580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 834/1652] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:42 2022] POST /identity/v3/users => generated 291 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:42.733873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e066f-0af5-4b7b-9bbb-a0b36a9c7195 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:42.753472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e066f-0af5-4b7b-9bbb-a0b36a9c7195 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:42.769159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a508847-5d1c-40c9-844f-5e0b427284eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:42.769756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a508847-5d1c-40c9-844f-5e0b427284eb admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:42.771511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a508847-5d1c-40c9-844f-5e0b427284eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:42.772040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a508847-5d1c-40c9-844f-5e0b427284eb admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:42.772935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a508847-5d1c-40c9-844f-5e0b427284eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:42.774661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a508847-5d1c-40c9-844f-5e0b427284eb admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:42.807826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 819/1653] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:42.812025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b96d5b-a7e1-419b-870d-7c0a9edc4730 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:42.832160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b96d5b-a7e1-419b-870d-7c0a9edc4730 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:42.848556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0af5b8-2b09-4ff9-88ef-0ad702c86c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:42.849226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0af5b8-2b09-4ff9-88ef-0ad702c86c65 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:42.849546 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0af5b8-2b09-4ff9-88ef-0ad702c86c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:42.849841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0af5b8-2b09-4ff9-88ef-0ad702c86c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:42.850809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0af5b8-2b09-4ff9-88ef-0ad702c86c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:42.852334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0af5b8-2b09-4ff9-88ef-0ad702c86c65 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:42.882299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 835/1654] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:42.887847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a508847-5d1c-40c9-844f-5e0b427284eb admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:42.915409 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a508847-5d1c-40c9-844f-5e0b427284eb admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:42.932657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e339d2-9721-4384-bc4b-04562ee38d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:42.933437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e339d2-9721-4384-bc4b-04562ee38d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:42.933650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e339d2-9721-4384-bc4b-04562ee38d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:42.933956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e339d2-9721-4384-bc4b-04562ee38d3f admin admin] PATH_INFO: `/v3/projects/697302ad93ce4d86893f7086f8505a59/users/da7ce95e46314e559aa8fe33ce4fe2ae/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:42.966409 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e339d2-9721-4384-bc4b-04562ee38d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=697302ad93ce4d86893f7086f8505a59, user_id=da7ce95e46314e559aa8fe33ce4fe2ae, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:42.969754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e339d2-9721-4384-bc4b-04562ee38d3f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:43.017938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 820/1655] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:42 2022] PUT /identity/v3/projects/697302ad93ce4d86893f7086f8505a59/users/da7ce95e46314e559aa8fe33ce4fe2ae/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:43.035666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb621a0-6c80-4f68-a83a-306db1f1a14e None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:43.036101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb621a0-6c80-4f68-a83a-306db1f1a14e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:43.036401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb621a0-6c80-4f68-a83a-306db1f1a14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:43.085437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-afb621a0-6c80-4f68-a83a-306db1f1a14e None None] MFA Rules not processed for user `da7ce95e46314e559aa8fe33ce4fe2ae`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:43.088629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-afb621a0-6c80-4f68-a83a-306db1f1a14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:43.285759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e339d2-9721-4384-bc4b-04562ee38d3f admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:43.286474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 836/1656] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:43.389607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:43.428982 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e339d2-9721-4384-bc4b-04562ee38d3f admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:43.435171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2201b-c563-4991-93fb-479dc601dadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:43.435799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2201b-c563-4991-93fb-479dc601dadd admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:43.436079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2201b-c563-4991-93fb-479dc601dadd admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:43.436387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2201b-c563-4991-93fb-479dc601dadd admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:43.437042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2201b-c563-4991-93fb-479dc601dadd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:43.439047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2201b-c563-4991-93fb-479dc601dadd admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:43.504255 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb621a0-6c80-4f68-a83a-306db1f1a14e None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:43.514667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 821/1657] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:43 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:43.528566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb621a0-6c80-4f68-a83a-306db1f1a14e None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:43.638289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33368d6-8426-48ce-9842-e582940c4370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:43.638925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33368d6-8426-48ce-9842-e582940c4370 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:43.639191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33368d6-8426-48ce-9842-e582940c4370 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:43.644718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33368d6-8426-48ce-9842-e582940c4370 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:43.645490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33368d6-8426-48ce-9842-e582940c4370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:43.647491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33368d6-8426-48ce-9842-e582940c4370 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:43.691043 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 837/1658] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:43 2022] POST /identity/v3/users => generated 296 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:45:43.695971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2201b-c563-4991-93fb-479dc601dadd admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:43.714449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2201b-c563-4991-93fb-479dc601dadd admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:43.724316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97d9e4-ecb0-41db-aefe-935d97861d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:43.724963 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97d9e4-ecb0-41db-aefe-935d97861d3a admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:43.725229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97d9e4-ecb0-41db-aefe-935d97861d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:43.725499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97d9e4-ecb0-41db-aefe-935d97861d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:43.726226 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97d9e4-ecb0-41db-aefe-935d97861d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:43.727853 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97d9e4-ecb0-41db-aefe-935d97861d3a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:43.748322 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 822/1659] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:43.752497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33368d6-8426-48ce-9842-e582940c4370 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:43.765749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33368d6-8426-48ce-9842-e582940c4370 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:43.779144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7889706-ff93-4cd2-88d0-b78e2e494184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:43.779677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7889706-ff93-4cd2-88d0-b78e2e494184 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:43.779958 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7889706-ff93-4cd2-88d0-b78e2e494184 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:43.780335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7889706-ff93-4cd2-88d0-b78e2e494184 admin admin] PATH_INFO: `/v3/projects/0852e65547af485ea79621066bbcc0e6/users/8de6e11a2d394bff947c4776a090dc33/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:43.826255 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7889706-ff93-4cd2-88d0-b78e2e494184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0852e65547af485ea79621066bbcc0e6, user_id=8de6e11a2d394bff947c4776a090dc33, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:43.829316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7889706-ff93-4cd2-88d0-b78e2e494184 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:43.836786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:43.863662 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 838/1660] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:43 2022] PUT /identity/v3/projects/0852e65547af485ea79621066bbcc0e6/users/8de6e11a2d394bff947c4776a090dc33/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:43.868529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97d9e4-ecb0-41db-aefe-935d97861d3a admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:43.882635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97d9e4-ecb0-41db-aefe-935d97861d3a admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:44.038947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfea96-77d8-4c0d-a646-dc474227854c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:44.039703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfea96-77d8-4c0d-a646-dc474227854c admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:44.040082 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfea96-77d8-4c0d-a646-dc474227854c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:44.040972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfea96-77d8-4c0d-a646-dc474227854c admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:44.041910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfea96-77d8-4c0d-a646-dc474227854c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:44.043634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfea96-77d8-4c0d-a646-dc474227854c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:44.054535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 823/1661] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:44.060575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7889706-ff93-4cd2-88d0-b78e2e494184 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:44.073764 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7889706-ff93-4cd2-88d0-b78e2e494184 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:44.085748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995efac2-e219-4569-81ee-dfdc0c78741a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:44.086651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995efac2-e219-4569-81ee-dfdc0c78741a admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:44.087064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995efac2-e219-4569-81ee-dfdc0c78741a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:44.087532 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995efac2-e219-4569-81ee-dfdc0c78741a admin admin] PATH_INFO: `/v3/projects/0852e65547af485ea79621066bbcc0e6/users/8de6e11a2d394bff947c4776a090dc33/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:44.098544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995efac2-e219-4569-81ee-dfdc0c78741a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0852e65547af485ea79621066bbcc0e6, user_id=8de6e11a2d394bff947c4776a090dc33, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:44.101727 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995efac2-e219-4569-81ee-dfdc0c78741a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:44.116318 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 839/1662] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:44 2022] PUT /identity/v3/projects/0852e65547af485ea79621066bbcc0e6/users/8de6e11a2d394bff947c4776a090dc33/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:44.129444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465fd5d-42b8-4351-bed5-a481c988135a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:44.129967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465fd5d-42b8-4351-bed5-a481c988135a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:44.130436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465fd5d-42b8-4351-bed5-a481c988135a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:44.193074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c465fd5d-42b8-4351-bed5-a481c988135a None None] MFA Rules not processed for user `8de6e11a2d394bff947c4776a090dc33`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:44.194734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c465fd5d-42b8-4351-bed5-a481c988135a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:44.371644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 824/1663] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:44.384209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995efac2-e219-4569-81ee-dfdc0c78741a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:44.386340 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-995efac2-e219-4569-81ee-dfdc0c78741a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:44.578905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995efac2-e219-4569-81ee-dfdc0c78741a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:44.593184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13fe45-fcd6-46e7-9fdd-ce2de21c2961 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:44.593854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13fe45-fcd6-46e7-9fdd-ce2de21c2961 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:44.594136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13fe45-fcd6-46e7-9fdd-ce2de21c2961 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:44.594410 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13fe45-fcd6-46e7-9fdd-ce2de21c2961 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:44.616533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13fe45-fcd6-46e7-9fdd-ce2de21c2961 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:44.621750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13fe45-fcd6-46e7-9fdd-ce2de21c2961 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:44.649770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 840/1664] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:44.800229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8d0a2-32e1-47a0-8784-8633d4285ba3 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:44.803853 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8d0a2-32e1-47a0-8784-8633d4285ba3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:44.804448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8d0a2-32e1-47a0-8784-8633d4285ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:44.891362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-bfb8d0a2-32e1-47a0-8784-8633d4285ba3 None None] MFA Rules not processed for user `3c5c5182fa3e46cab4759d2782feb98e`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:44.892826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-bfb8d0a2-32e1-47a0-8784-8633d4285ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:44.958884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:45.061919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 825/1665] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:44 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:45.625980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13fe45-fcd6-46e7-9fdd-ce2de21c2961 service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:45.776000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13fe45-fcd6-46e7-9fdd-ce2de21c2961 service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:45.787943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:45.788620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:45.788897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:45.789173 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] PATH_INFO: `/v3/users/da7ce95e46314e559aa8fe33ce4fe2ae` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:45.793032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da7ce95e46314e559aa8fe33ce4fe2ae)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:45.794136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: WARNING py.warnings [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 08:45:45.794136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: warnings.warn(msg) Oct 04 08:45:45.794136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: Oct 04 08:45:45.800017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:45.922404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da7ce95e46314e559aa8fe33ce4fe2ae'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:45.923158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da7ce95e46314e559aa8fe33ce4fe2ae'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:45.945901 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da7ce95e46314e559aa8fe33ce4fe2ae'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:45.947311 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da7ce95e46314e559aa8fe33ce4fe2ae'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:45.990800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da7ce95e46314e559aa8fe33ce4fe2ae'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:46.016976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 841/1666] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:45:45 2022] DELETE /identity/v3/users/da7ce95e46314e559aa8fe33ce4fe2ae => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:45:46.021934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428232c-f9ed-45b6-a801-2caf393df88d None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:46.022987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428232c-f9ed-45b6-a801-2caf393df88d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:46.023454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428232c-f9ed-45b6-a801-2caf393df88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:46.076106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c428232c-f9ed-45b6-a801-2caf393df88d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:46.077905 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c428232c-f9ed-45b6-a801-2caf393df88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:46.192428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 826/1667] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:46.250360 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:46.252760 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:46.261611 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:46.408547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b899c-345b-4cd1-a897-f5796802bde9 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:46.417316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3db68-8e80-4e19-9a68-ac4bc52d9fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:46.418178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3db68-8e80-4e19-9a68-ac4bc52d9fbf service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:46.418573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3db68-8e80-4e19-9a68-ac4bc52d9fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:46.419408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3db68-8e80-4e19-9a68-ac4bc52d9fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:46.443059 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3db68-8e80-4e19-9a68-ac4bc52d9fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:46.445321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3db68-8e80-4e19-9a68-ac4bc52d9fbf service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:46.467653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 842/1668] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:47.181369 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428232c-f9ed-45b6-a801-2caf393df88d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:47.183454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c428232c-f9ed-45b6-a801-2caf393df88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:47.338916 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428232c-f9ed-45b6-a801-2caf393df88d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:47.352575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f401e-4117-46bf-8072-2ed7604b175b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:47.353489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f401e-4117-46bf-8072-2ed7604b175b service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:47.353946 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f401e-4117-46bf-8072-2ed7604b175b service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:47.354349 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f401e-4117-46bf-8072-2ed7604b175b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:47.356551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ae7f401e-4117-46bf-8072-2ed7604b175b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:47.440360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f401e-4117-46bf-8072-2ed7604b175b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:47.445620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7f401e-4117-46bf-8072-2ed7604b175b service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:47.464907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 827/1669] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:47.723130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3db68-8e80-4e19-9a68-ac4bc52d9fbf service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:47.725392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-83b3db68-8e80-4e19-9a68-ac4bc52d9fbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:47.769447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3db68-8e80-4e19-9a68-ac4bc52d9fbf service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:47.788265 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:47.789663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:47.792692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:47.793256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4 admin admin] PATH_INFO: `/v3/projects/697302ad93ce4d86893f7086f8505a59` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:47.798989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=697302ad93ce4d86893f7086f8505a59)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:47.801084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:47.925288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '697302ad93ce4d86893f7086f8505a59', 'request_id': 'req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:47.935406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '697302ad93ce4d86893f7086f8505a59', 'request_id': 'req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:47.950575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '697302ad93ce4d86893f7086f8505a59', 'request_id': 'req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:47.982930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4 admin admin] The token cache is being invalidate because project 697302ad93ce4d86893f7086f8505a59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:45:47.983412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:47.984737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ceb9ab90-05b7-437f-a30c-f3ce7037f8f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:47.990892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 843/1670] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:45:47 2022] DELETE /identity/v3/projects/697302ad93ce4d86893f7086f8505a59 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:47.995147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f401e-4117-46bf-8072-2ed7604b175b service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:48.011992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f401e-4117-46bf-8072-2ed7604b175b service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:48.046855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:48.200472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b1892-3663-4937-808b-c413c063d283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:48.201289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b1892-3663-4937-808b-c413c063d283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:48.203110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b1892-3663-4937-808b-c413c063d283 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:48.204419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b1892-3663-4937-808b-c413c063d283 admin admin] PATH_INFO: `/v3/users/8de6e11a2d394bff947c4776a090dc33` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:48.208350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b1892-3663-4937-808b-c413c063d283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de6e11a2d394bff947c4776a090dc33)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:48.213840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b1892-3663-4937-808b-c413c063d283 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:48.330935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-0c9b1892-3663-4937-808b-c413c063d283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de6e11a2d394bff947c4776a090dc33'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:48.331985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-0c9b1892-3663-4937-808b-c413c063d283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de6e11a2d394bff947c4776a090dc33'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:48.360875 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-0c9b1892-3663-4937-808b-c413c063d283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de6e11a2d394bff947c4776a090dc33'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:48.380597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-0c9b1892-3663-4937-808b-c413c063d283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de6e11a2d394bff947c4776a090dc33'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:48.381783 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-0c9b1892-3663-4937-808b-c413c063d283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de6e11a2d394bff947c4776a090dc33'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:48.414469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 828/1671] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:45:47 2022] DELETE /identity/v3/users/8de6e11a2d394bff947c4776a090dc33 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:48.694707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71ade0-6b7f-4018-9fca-f95ef88edde8 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:48.694994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71ade0-6b7f-4018-9fca-f95ef88edde8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:48.695821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71ade0-6b7f-4018-9fca-f95ef88edde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:48.760506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6a71ade0-6b7f-4018-9fca-f95ef88edde8 None None] MFA Rules not processed for user `980f097960df41528ac45d6c5a07c5f5`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:48.761888 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6a71ade0-6b7f-4018-9fca-f95ef88edde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:48.939428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 844/1672] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:48.951939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1e14b-a557-4f46-a13e-73de1759a8e7 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:48.952088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1e14b-a557-4f46-a13e-73de1759a8e7 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:48.952310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1e14b-a557-4f46-a13e-73de1759a8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:48.991962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-10d1e14b-a557-4f46-a13e-73de1759a8e7 None None] MFA Rules not processed for user `980f097960df41528ac45d6c5a07c5f5`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:48.993215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-10d1e14b-a557-4f46-a13e-73de1759a8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:49.014796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 829/1673] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:49.026466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71ade0-6b7f-4018-9fca-f95ef88edde8 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:49.028106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6a71ade0-6b7f-4018-9fca-f95ef88edde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:49.067940 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:49.108753 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1e14b-a557-4f46-a13e-73de1759a8e7 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:49.110428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-10d1e14b-a557-4f46-a13e-73de1759a8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:49.234026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a71ade0-6b7f-4018-9fca-f95ef88edde8 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:49.246092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de1a15-7a37-4539-aa38-8f680f230994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:49.247584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de1a15-7a37-4539-aa38-8f680f230994 service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:49.248061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de1a15-7a37-4539-aa38-8f680f230994 service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:49.248466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de1a15-7a37-4539-aa38-8f680f230994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:49.289332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1e14b-a557-4f46-a13e-73de1759a8e7 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:49.297287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b01d68-0398-45cc-81e8-bf53cd56c7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:49.299705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b01d68-0398-45cc-81e8-bf53cd56c7e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:49.300147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b01d68-0398-45cc-81e8-bf53cd56c7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:49.301061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b01d68-0398-45cc-81e8-bf53cd56c7e0 admin admin] PATH_INFO: `/v3/projects/0852e65547af485ea79621066bbcc0e6` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:49.302639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b01d68-0398-45cc-81e8-bf53cd56c7e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0852e65547af485ea79621066bbcc0e6)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:49.304649 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de1a15-7a37-4539-aa38-8f680f230994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:49.305752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b01d68-0398-45cc-81e8-bf53cd56c7e0 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:49.307309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de1a15-7a37-4539-aa38-8f680f230994 service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:49.336254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 845/1674] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:49.339391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c3809-0b88-42dc-81c0-a68c52f5853b None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:49.339857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c3809-0b88-42dc-81c0-a68c52f5853b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:49.340271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c3809-0b88-42dc-81c0-a68c52f5853b None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:49.342048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 846/1675] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:45:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:49.401111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:49.462972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-64b01d68-0398-45cc-81e8-bf53cd56c7e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0852e65547af485ea79621066bbcc0e6', 'request_id': 'req-64b01d68-0398-45cc-81e8-bf53cd56c7e0'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:49.470321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaac0a2-58e2-40bd-8505-8bc2e2ce1b2f None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:49.471371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaac0a2-58e2-40bd-8505-8bc2e2ce1b2f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:49.471657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaac0a2-58e2-40bd-8505-8bc2e2ce1b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:49.510343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-64b01d68-0398-45cc-81e8-bf53cd56c7e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0852e65547af485ea79621066bbcc0e6', 'request_id': 'req-64b01d68-0398-45cc-81e8-bf53cd56c7e0'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:49.529818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-cfaac0a2-58e2-40bd-8505-8bc2e2ce1b2f None None] MFA Rules not processed for user `abd158648914409f87ce0599843f0fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:49.532610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-64b01d68-0398-45cc-81e8-bf53cd56c7e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0852e65547af485ea79621066bbcc0e6', 'request_id': 'req-64b01d68-0398-45cc-81e8-bf53cd56c7e0'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:49.534468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-cfaac0a2-58e2-40bd-8505-8bc2e2ce1b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:49.559217 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-64b01d68-0398-45cc-81e8-bf53cd56c7e0 admin admin] The token cache is being invalidate because project 0852e65547af485ea79621066bbcc0e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:45:49.560309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-64b01d68-0398-45cc-81e8-bf53cd56c7e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:49.564375 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-64b01d68-0398-45cc-81e8-bf53cd56c7e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:49.569185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 830/1676] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:45:49 2022] DELETE /identity/v3/projects/0852e65547af485ea79621066bbcc0e6 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:49.575568 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e3746-5fae-4b50-be31-3fe56c7423db None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:49.576347 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e3746-5fae-4b50-be31-3fe56c7423db None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:49.577242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e3746-5fae-4b50-be31-3fe56c7423db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:49.630874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-462e3746-5fae-4b50-be31-3fe56c7423db None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:49.632398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-462e3746-5fae-4b50-be31-3fe56c7423db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:49.712639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 847/1677] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:45:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:49.793540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 831/1678] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:49.796067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547df18d-d0a8-43aa-b250-c98a099f1d7f None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:49.796647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547df18d-d0a8-43aa-b250-c98a099f1d7f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:49.797011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547df18d-d0a8-43aa-b250-c98a099f1d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:49.818095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e3746-5fae-4b50-be31-3fe56c7423db None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:49.819675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-462e3746-5fae-4b50-be31-3fe56c7423db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:49.850107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e3746-5fae-4b50-be31-3fe56c7423db None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:49.860031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3775922-2037-44ed-a6e2-61e9b1d505d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:49.860838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3775922-2037-44ed-a6e2-61e9b1d505d8 service placement] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:49.861228 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3775922-2037-44ed-a6e2-61e9b1d505d8 service placement] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:49.861616 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3775922-2037-44ed-a6e2-61e9b1d505d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:49.874938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-547df18d-d0a8-43aa-b250-c98a099f1d7f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:49.877455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-547df18d-d0a8-43aa-b250-c98a099f1d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:49.882597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3775922-2037-44ed-a6e2-61e9b1d505d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:49.887107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3775922-2037-44ed-a6e2-61e9b1d505d8 service placement] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:49.894537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 848/1679] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:49.899344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547df18d-d0a8-43aa-b250-c98a099f1d7f None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:49.913481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 832/1680] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:45:49.938147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547df18d-d0a8-43aa-b250-c98a099f1d7f None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:49.955892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99890b95-eed0-497b-a163-7158084ce8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:49.956732 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99890b95-eed0-497b-a163-7158084ce8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:49.957147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99890b95-eed0-497b-a163-7158084ce8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:49.957554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99890b95-eed0-497b-a163-7158084ce8a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:49.958381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99890b95-eed0-497b-a163-7158084ce8a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:45:49.959768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99890b95-eed0-497b-a163-7158084ce8a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:49.974948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99890b95-eed0-497b-a163-7158084ce8a3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:49.995568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 849/1681] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:45:50.000978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3775922-2037-44ed-a6e2-61e9b1d505d8 service placement] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:50.017387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3775922-2037-44ed-a6e2-61e9b1d505d8 service placement] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:50.022770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fb88a-bfea-4501-9815-d4312d78deee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:50.023392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fb88a-bfea-4501-9815-d4312d78deee admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:50.023675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fb88a-bfea-4501-9815-d4312d78deee admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:50.023950 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fb88a-bfea-4501-9815-d4312d78deee admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:50.024632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fb88a-bfea-4501-9815-d4312d78deee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:50.026602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fb88a-bfea-4501-9815-d4312d78deee admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:50.101563 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 833/1682] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:49 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:50.108481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99890b95-eed0-497b-a163-7158084ce8a3 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:50.122526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99890b95-eed0-497b-a163-7158084ce8a3 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:50.253588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f41cb6-917b-4258-b9b7-48ce5203cdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:50.254411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f41cb6-917b-4258-b9b7-48ce5203cdc1 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:50.254526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f41cb6-917b-4258-b9b7-48ce5203cdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:50.254771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f41cb6-917b-4258-b9b7-48ce5203cdc1 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:50.255414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f41cb6-917b-4258-b9b7-48ce5203cdc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:50.257768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f41cb6-917b-4258-b9b7-48ce5203cdc1 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:50.315661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 850/1683] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:50 2022] POST /identity/v3/users => generated 286 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:50.322064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fb88a-bfea-4501-9815-d4312d78deee admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:50.349936 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fb88a-bfea-4501-9815-d4312d78deee admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:50.361812 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9e88d-36fa-4320-b976-95bc8feb9ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:50.362572 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9e88d-36fa-4320-b976-95bc8feb9ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:50.362956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9e88d-36fa-4320-b976-95bc8feb9ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:50.363368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9e88d-36fa-4320-b976-95bc8feb9ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:50.365095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9e88d-36fa-4320-b976-95bc8feb9ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:50.386396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9e88d-36fa-4320-b976-95bc8feb9ae4 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:50.408336 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 834/1684] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:50.411708 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f41cb6-917b-4258-b9b7-48ce5203cdc1 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:50.432944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f41cb6-917b-4258-b9b7-48ce5203cdc1 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:50.443540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042e7f8-9a91-4e7a-8e0c-f5bd642f86e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:50.445663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042e7f8-9a91-4e7a-8e0c-f5bd642f86e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:50.446074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042e7f8-9a91-4e7a-8e0c-f5bd642f86e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:50.456603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042e7f8-9a91-4e7a-8e0c-f5bd642f86e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:50.457620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042e7f8-9a91-4e7a-8e0c-f5bd642f86e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:50.459554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042e7f8-9a91-4e7a-8e0c-f5bd642f86e0 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:50.474863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:50.507505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 851/1685] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:50.512833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9e88d-36fa-4320-b976-95bc8feb9ae4 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:50.533653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9e88d-36fa-4320-b976-95bc8feb9ae4 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:50.546626 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee1d1e-0ec1-4d39-be3e-d11883f13bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:50.547408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee1d1e-0ec1-4d39-be3e-d11883f13bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:50.547799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee1d1e-0ec1-4d39-be3e-d11883f13bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:50.548174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee1d1e-0ec1-4d39-be3e-d11883f13bb1 admin admin] PATH_INFO: `/v3/projects/baa9503eacbd4bf18227740b51a60225/users/e59561752faa47c19dbb56a196423b02/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:50.592326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee1d1e-0ec1-4d39-be3e-d11883f13bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baa9503eacbd4bf18227740b51a60225, user_id=e59561752faa47c19dbb56a196423b02, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:50.595194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee1d1e-0ec1-4d39-be3e-d11883f13bb1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:50.627622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 835/1686] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:50 2022] PUT /identity/v3/projects/baa9503eacbd4bf18227740b51a60225/users/e59561752faa47c19dbb56a196423b02/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:50.640330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61698380-32a1-46e5-abfa-85a6b0117779 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:50.640756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61698380-32a1-46e5-abfa-85a6b0117779 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:50.641269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61698380-32a1-46e5-abfa-85a6b0117779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:50.703877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-61698380-32a1-46e5-abfa-85a6b0117779 None None] MFA Rules not processed for user `e59561752faa47c19dbb56a196423b02`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:50.705426 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-61698380-32a1-46e5-abfa-85a6b0117779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:50.765020 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251b53c-b3b4-455e-a95d-eeee7925a104 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:50.765450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251b53c-b3b4-455e-a95d-eeee7925a104 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:50.765863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251b53c-b3b4-455e-a95d-eeee7925a104 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:50.767832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 836/1687] 10.222.0.26 () {54 vars in 912 bytes} [Tue Oct 4 08:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:50.775150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7413487-c3ad-48a3-96fe-d06af7f7019f None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:50.780564 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7413487-c3ad-48a3-96fe-d06af7f7019f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:50.780564 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7413487-c3ad-48a3-96fe-d06af7f7019f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:50.886402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-b7413487-c3ad-48a3-96fe-d06af7f7019f None None] MFA Rules not processed for user `50d3b75cbc5c4d1686f4d93248e04746`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:50.888040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b7413487-c3ad-48a3-96fe-d06af7f7019f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:50.895169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 852/1688] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:50.897414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e88b07-176f-4dae-8ed1-689458bae401 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:50.897881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e88b07-176f-4dae-8ed1-689458bae401 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:50.898297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e88b07-176f-4dae-8ed1-689458bae401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:50.957864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:51.004147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b5e88b07-176f-4dae-8ed1-689458bae401 None None] MFA Rules not processed for user `ed4b7bfb87b144b38b82514c17a468bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:51.006251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b5e88b07-176f-4dae-8ed1-689458bae401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:51.017600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 837/1689] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:45:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:51.020004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7413487-c3ad-48a3-96fe-d06af7f7019f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:51.027576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b7413487-c3ad-48a3-96fe-d06af7f7019f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:51.154638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7413487-c3ad-48a3-96fe-d06af7f7019f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:51.157543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 853/1690] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:51.170507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38efd1-f1ab-42b6-b045-b948ae3221fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:51.172767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38efd1-f1ab-42b6-b045-b948ae3221fa service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:51.174426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38efd1-f1ab-42b6-b045-b948ae3221fa service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:51.174985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38efd1-f1ab-42b6-b045-b948ae3221fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:51.176738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e88b07-176f-4dae-8ed1-689458bae401 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:51.179518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b5e88b07-176f-4dae-8ed1-689458bae401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:51.197333 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38efd1-f1ab-42b6-b045-b948ae3221fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:51.201074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38efd1-f1ab-42b6-b045-b948ae3221fa service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:51.233356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 838/1691] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:51.260489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38efd1-f1ab-42b6-b045-b948ae3221fa service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:51.401388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e88b07-176f-4dae-8ed1-689458bae401 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:51.417227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f5243-81f5-4bff-baaf-796fd6912a5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:51.417893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f5243-81f5-4bff-baaf-796fd6912a5c service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:51.418177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f5243-81f5-4bff-baaf-796fd6912a5c service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:51.418451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f5243-81f5-4bff-baaf-796fd6912a5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:51.421232 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38efd1-f1ab-42b6-b045-b948ae3221fa service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:51.434101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d978d8d-00db-4083-8737-5a3d0652180b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:51.436517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d978d8d-00db-4083-8737-5a3d0652180b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:51.436635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d978d8d-00db-4083-8737-5a3d0652180b admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:51.436845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d978d8d-00db-4083-8737-5a3d0652180b admin admin] PATH_INFO: `/v3/users/e59561752faa47c19dbb56a196423b02` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:51.438698 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f5243-81f5-4bff-baaf-796fd6912a5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:51.439570 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d978d8d-00db-4083-8737-5a3d0652180b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59561752faa47c19dbb56a196423b02)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:51.440338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f5243-81f5-4bff-baaf-796fd6912a5c service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:51.445289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d978d8d-00db-4083-8737-5a3d0652180b admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:51.485107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 854/1692] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:51.486124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f5243-81f5-4bff-baaf-796fd6912a5c service nova] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:51.489571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f44f5243-81f5-4bff-baaf-796fd6912a5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:51.518711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:51.547889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f5243-81f5-4bff-baaf-796fd6912a5c service nova] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:51.553261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce3614-9acd-4052-a558-361fa2d8a168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:51.553933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce3614-9acd-4052-a558-361fa2d8a168 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:51.554215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce3614-9acd-4052-a558-361fa2d8a168 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:51.554489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce3614-9acd-4052-a558-361fa2d8a168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:51.565967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4d978d8d-00db-4083-8737-5a3d0652180b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59561752faa47c19dbb56a196423b02'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:51.565967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4d978d8d-00db-4083-8737-5a3d0652180b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59561752faa47c19dbb56a196423b02'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:51.578733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce3614-9acd-4052-a558-361fa2d8a168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:51.580290 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce3614-9acd-4052-a558-361fa2d8a168 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:51.599614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 855/1693] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:51.600979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196536f-3363-47d9-9bca-8f454833902e None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:51.601263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196536f-3363-47d9-9bca-8f454833902e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:51.601543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196536f-3363-47d9-9bca-8f454833902e None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:51.606343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 856/1694] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:45:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:45:51.607610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4d978d8d-00db-4083-8737-5a3d0652180b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59561752faa47c19dbb56a196423b02'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:51.628918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4d978d8d-00db-4083-8737-5a3d0652180b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59561752faa47c19dbb56a196423b02'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:51.633510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4d978d8d-00db-4083-8737-5a3d0652180b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59561752faa47c19dbb56a196423b02'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:51.661967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 839/1695] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:45:51 2022] DELETE /identity/v3/users/e59561752faa47c19dbb56a196423b02 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:51.666727 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2b7dc-3866-48bd-a7d3-b4675bd69ea8 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:51.667021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2b7dc-3866-48bd-a7d3-b4675bd69ea8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:51.667298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2b7dc-3866-48bd-a7d3-b4675bd69ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:51.685857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e9954-fbac-4701-a01f-906aa9ca7390 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:51.686567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e9954-fbac-4701-a01f-906aa9ca7390 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:51.686856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e9954-fbac-4701-a01f-906aa9ca7390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:51.706927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:51.722428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-1fa2b7dc-3866-48bd-a7d3-b4675bd69ea8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:51.722860 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1fa2b7dc-3866-48bd-a7d3-b4675bd69ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:51.742720 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-216e9954-fbac-4701-a01f-906aa9ca7390 None None] MFA Rules not processed for user `abd158648914409f87ce0599843f0fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:51.744133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-216e9954-fbac-4701-a01f-906aa9ca7390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:51.867175 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 840/1696] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:45:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:45:51.868126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e9954-fbac-4701-a01f-906aa9ca7390 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:51.883555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-216e9954-fbac-4701-a01f-906aa9ca7390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:51.893793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 857/1697] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:51.915944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2b7dc-3866-48bd-a7d3-b4675bd69ea8 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:51.917627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1fa2b7dc-3866-48bd-a7d3-b4675bd69ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:51.924129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e9954-fbac-4701-a01f-906aa9ca7390 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:51.931335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:51.932192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:51.933813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:51.934235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] PATH_INFO: `/v3/users/8532c435f485429bb5b9acc29236cfbb` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:51.936220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8532c435f485429bb5b9acc29236cfbb)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:51.939877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:51.962927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2b7dc-3866-48bd-a7d3-b4675bd69ea8 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:51.975534 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cebbd-f6c6-4f55-9302-c6627b321327 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:51.976239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cebbd-f6c6-4f55-9302-c6627b321327 service placement] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:51.976477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cebbd-f6c6-4f55-9302-c6627b321327 service placement] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:51.976744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cebbd-f6c6-4f55-9302-c6627b321327 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:52.001929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cebbd-f6c6-4f55-9302-c6627b321327 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:52.003570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cebbd-f6c6-4f55-9302-c6627b321327 service placement] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:52.010948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8532c435f485429bb5b9acc29236cfbb'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:52.011972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8532c435f485429bb5b9acc29236cfbb'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:52.025935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 858/1698] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:52.026826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cebbd-f6c6-4f55-9302-c6627b321327 service placement] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:52.028268 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-681cebbd-f6c6-4f55-9302-c6627b321327 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:52.056851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8532c435f485429bb5b9acc29236cfbb'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:52.068005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8532c435f485429bb5b9acc29236cfbb'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:52.069039 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8532c435f485429bb5b9acc29236cfbb'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:52.114980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 841/1699] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:45:51 2022] DELETE /identity/v3/users/8532c435f485429bb5b9acc29236cfbb => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:52.187898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cebbd-f6c6-4f55-9302-c6627b321327 service placement] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:52.312019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5faea-51df-4f6c-9892-a243af61c984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:52.312019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5faea-51df-4f6c-9892-a243af61c984 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:52.312019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5faea-51df-4f6c-9892-a243af61c984 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:52.312019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5faea-51df-4f6c-9892-a243af61c984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:52.314232 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-46f5faea-51df-4f6c-9892-a243af61c984 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:52.437755 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:52.474653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5faea-51df-4f6c-9892-a243af61c984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:52.476151 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5faea-51df-4f6c-9892-a243af61c984 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:52.498034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 859/1700] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:53.047793 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:53.049333 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:53.183478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3582a3-5dab-4310-8acf-a5b952dd757c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:53.192719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957eee76-a1b0-4887-88b7-3880eb118d72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:53.193463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957eee76-a1b0-4887-88b7-3880eb118d72 service placement] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:53.196240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957eee76-a1b0-4887-88b7-3880eb118d72 service placement] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:53.197220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957eee76-a1b0-4887-88b7-3880eb118d72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:53.199714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-957eee76-a1b0-4887-88b7-3880eb118d72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:53.218959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957eee76-a1b0-4887-88b7-3880eb118d72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:53.221366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957eee76-a1b0-4887-88b7-3880eb118d72 service placement] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:53.249747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 842/1701] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:53.569505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5faea-51df-4f6c-9892-a243af61c984 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:53.571101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-46f5faea-51df-4f6c-9892-a243af61c984 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:53.606684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5faea-51df-4f6c-9892-a243af61c984 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:53.632862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147cc16-e494-4271-a89d-879e3cc7d701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:53.633526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147cc16-e494-4271-a89d-879e3cc7d701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:53.633919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147cc16-e494-4271-a89d-879e3cc7d701 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:53.634200 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147cc16-e494-4271-a89d-879e3cc7d701 admin admin] PATH_INFO: `/v3/projects/60c4101d507d4b618cf45d535eb6170c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:53.636585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5147cc16-e494-4271-a89d-879e3cc7d701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60c4101d507d4b618cf45d535eb6170c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:53.638565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5147cc16-e494-4271-a89d-879e3cc7d701 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:53.785774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5147cc16-e494-4271-a89d-879e3cc7d701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60c4101d507d4b618cf45d535eb6170c', 'request_id': 'req-5147cc16-e494-4271-a89d-879e3cc7d701'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:53.798542 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5147cc16-e494-4271-a89d-879e3cc7d701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60c4101d507d4b618cf45d535eb6170c', 'request_id': 'req-5147cc16-e494-4271-a89d-879e3cc7d701'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:53.813223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5147cc16-e494-4271-a89d-879e3cc7d701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60c4101d507d4b618cf45d535eb6170c', 'request_id': 'req-5147cc16-e494-4271-a89d-879e3cc7d701'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:53.840767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5147cc16-e494-4271-a89d-879e3cc7d701 admin admin] The token cache is being invalidate because project 60c4101d507d4b618cf45d535eb6170c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:45:53.841074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5147cc16-e494-4271-a89d-879e3cc7d701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:53.842342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5147cc16-e494-4271-a89d-879e3cc7d701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:53.848080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 860/1702] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:45:53 2022] DELETE /identity/v3/projects/60c4101d507d4b618cf45d535eb6170c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:53.849103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957eee76-a1b0-4887-88b7-3880eb118d72 service placement] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:53.856252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:53.877019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957eee76-a1b0-4887-88b7-3880eb118d72 service placement] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:54.028187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75901eed-64a6-4c1b-9489-17bc0c22ff2c None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:54.028655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75901eed-64a6-4c1b-9489-17bc0c22ff2c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:54.029099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75901eed-64a6-4c1b-9489-17bc0c22ff2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:54.050545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:54.053340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:54.053758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:54.054044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] PATH_INFO: `/v3/users/3c5c5182fa3e46cab4759d2782feb98e` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:54.063554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c5c5182fa3e46cab4759d2782feb98e)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:54.069083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:54.146414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-75901eed-64a6-4c1b-9489-17bc0c22ff2c None None] MFA Rules not processed for user `3c0b7528b0ff4b09923a0e951d052ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:54.148149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-75901eed-64a6-4c1b-9489-17bc0c22ff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:54.202531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c5c5182fa3e46cab4759d2782feb98e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:54.203324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c5c5182fa3e46cab4759d2782feb98e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:54.233731 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c5c5182fa3e46cab4759d2782feb98e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:54.265591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c5c5182fa3e46cab4759d2782feb98e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:54.270594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c5c5182fa3e46cab4759d2782feb98e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:54.312046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 843/1703] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:45:53 2022] DELETE /identity/v3/users/3c5c5182fa3e46cab4759d2782feb98e => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:45:54.314389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:54.324880 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:54.425820 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 861/1704] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:54.440414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75901eed-64a6-4c1b-9489-17bc0c22ff2c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:54.442213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-75901eed-64a6-4c1b-9489-17bc0c22ff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:54.621152 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6fc6f-8818-4372-a4b0-b00bb645fe35 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:54.643536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701294c-b285-4712-9177-4db619cce1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:54.644212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701294c-b285-4712-9177-4db619cce1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:54.644497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701294c-b285-4712-9177-4db619cce1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:54.644772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701294c-b285-4712-9177-4db619cce1a2 admin admin] PATH_INFO: `/v3/projects/baa9503eacbd4bf18227740b51a60225` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:54.646741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701294c-b285-4712-9177-4db619cce1a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baa9503eacbd4bf18227740b51a60225)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:54.653401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701294c-b285-4712-9177-4db619cce1a2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:54.653541 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:54.707187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75901eed-64a6-4c1b-9489-17bc0c22ff2c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:54.749103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:54.810028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1701294c-b285-4712-9177-4db619cce1a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baa9503eacbd4bf18227740b51a60225', 'request_id': 'req-1701294c-b285-4712-9177-4db619cce1a2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:54.827922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1701294c-b285-4712-9177-4db619cce1a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baa9503eacbd4bf18227740b51a60225', 'request_id': 'req-1701294c-b285-4712-9177-4db619cce1a2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:54.855876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1701294c-b285-4712-9177-4db619cce1a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baa9503eacbd4bf18227740b51a60225', 'request_id': 'req-1701294c-b285-4712-9177-4db619cce1a2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:54.902190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1701294c-b285-4712-9177-4db619cce1a2 admin admin] The token cache is being invalidate because project baa9503eacbd4bf18227740b51a60225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:45:54.904983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1701294c-b285-4712-9177-4db619cce1a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:54.906366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1701294c-b285-4712-9177-4db619cce1a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:54.910876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 844/1705] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:45:54 2022] DELETE /identity/v3/projects/baa9503eacbd4bf18227740b51a60225 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:54.916235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35351dfd-a4c3-455a-9f40-cc9208a0d39b None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:54.916525 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35351dfd-a4c3-455a-9f40-cc9208a0d39b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:54.916799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35351dfd-a4c3-455a-9f40-cc9208a0d39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:54.917301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f4a55-5052-48f8-9f1f-ef655d91b66c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:54.917987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f4a55-5052-48f8-9f1f-ef655d91b66c service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:54.918271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f4a55-5052-48f8-9f1f-ef655d91b66c service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:54.918587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f4a55-5052-48f8-9f1f-ef655d91b66c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:54.922696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1f6f4a55-5052-48f8-9f1f-ef655d91b66c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:54.964402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-35351dfd-a4c3-455a-9f40-cc9208a0d39b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:54.966811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-35351dfd-a4c3-455a-9f40-cc9208a0d39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:55.090755 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f4a55-5052-48f8-9f1f-ef655d91b66c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:55.092373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f4a55-5052-48f8-9f1f-ef655d91b66c service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:55.124882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 862/1706] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:55.125957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f4a55-5052-48f8-9f1f-ef655d91b66c service nova] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:55.131866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1f6f4a55-5052-48f8-9f1f-ef655d91b66c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:55.172589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 845/1707] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:45:55.175093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f4a55-5052-48f8-9f1f-ef655d91b66c service nova] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:55.184752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d5677-6e44-4c85-bc25-535af5e97c16 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:55.185350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d5677-6e44-4c85-bc25-535af5e97c16 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:55.185808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d5677-6e44-4c85-bc25-535af5e97c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:55.199580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e842e-cde5-4561-b09b-20f813b9c38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:55.200252 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e842e-cde5-4561-b09b-20f813b9c38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:55.200538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e842e-cde5-4561-b09b-20f813b9c38e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:55.200813 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e842e-cde5-4561-b09b-20f813b9c38e admin admin] PATH_INFO: `/v3/projects/24cece7ccdb746669c41342bd7f45305` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:55.207167 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e842e-cde5-4561-b09b-20f813b9c38e admin admin] RBAC: Authorizing `identity:delete_project(project_id=24cece7ccdb746669c41342bd7f45305)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:55.209139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e842e-cde5-4561-b09b-20f813b9c38e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:55.262456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-660d5677-6e44-4c85-bc25-535af5e97c16 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:55.263771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-660d5677-6e44-4c85-bc25-535af5e97c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:55.296861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 846/1708] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:55.301141 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d5677-6e44-4c85-bc25-535af5e97c16 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:55.322912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-734e842e-cde5-4561-b09b-20f813b9c38e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24cece7ccdb746669c41342bd7f45305', 'request_id': 'req-734e842e-cde5-4561-b09b-20f813b9c38e'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:55.326467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d5677-6e44-4c85-bc25-535af5e97c16 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:55.335937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-734e842e-cde5-4561-b09b-20f813b9c38e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24cece7ccdb746669c41342bd7f45305', 'request_id': 'req-734e842e-cde5-4561-b09b-20f813b9c38e'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:55.355269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-734e842e-cde5-4561-b09b-20f813b9c38e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24cece7ccdb746669c41342bd7f45305', 'request_id': 'req-734e842e-cde5-4561-b09b-20f813b9c38e'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:55.379716 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-734e842e-cde5-4561-b09b-20f813b9c38e admin admin] The token cache is being invalidate because project 24cece7ccdb746669c41342bd7f45305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:45:55.380015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-734e842e-cde5-4561-b09b-20f813b9c38e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:55.380824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-734e842e-cde5-4561-b09b-20f813b9c38e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:55.392271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 863/1709] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:45:55 2022] DELETE /identity/v3/projects/24cece7ccdb746669c41342bd7f45305 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:55.397668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e434e-15e5-4d94-bf29-d9f667e4110b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:55.397940 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e434e-15e5-4d94-bf29-d9f667e4110b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:55.398224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e434e-15e5-4d94-bf29-d9f667e4110b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:55.438232 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-e93e434e-15e5-4d94-bf29-d9f667e4110b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:55.440131 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e93e434e-15e5-4d94-bf29-d9f667e4110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:55.466436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:55.524235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e04ed-87f2-43ea-9f7f-495c9d840e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:55.525606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e04ed-87f2-43ea-9f7f-495c9d840e37 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:55.526477 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e04ed-87f2-43ea-9f7f-495c9d840e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:55.527072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e04ed-87f2-43ea-9f7f-495c9d840e37 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:55.528008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e04ed-87f2-43ea-9f7f-495c9d840e37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:45:55.528410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e04ed-87f2-43ea-9f7f-495c9d840e37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:55.531016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e04ed-87f2-43ea-9f7f-495c9d840e37 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:55.558899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 847/1710] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:45:55.563582 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e04ed-87f2-43ea-9f7f-495c9d840e37 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:55.567066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b11e04ed-87f2-43ea-9f7f-495c9d840e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:55.623830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e04ed-87f2-43ea-9f7f-495c9d840e37 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:55.648641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a632c-ff54-49af-a1fb-1f1a047d3752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:55.650270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a632c-ff54-49af-a1fb-1f1a047d3752 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:55.650270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a632c-ff54-49af-a1fb-1f1a047d3752 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:55.650462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a632c-ff54-49af-a1fb-1f1a047d3752 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:55.651006 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a632c-ff54-49af-a1fb-1f1a047d3752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:55.654303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a632c-ff54-49af-a1fb-1f1a047d3752 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:55.666923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 864/1711] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:55.669526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480a3c7-21b6-4e76-9ad9-c7bd86d73700 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:55.669869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480a3c7-21b6-4e76-9ad9-c7bd86d73700 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:55.670151 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480a3c7-21b6-4e76-9ad9-c7bd86d73700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:55.734874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-e480a3c7-21b6-4e76-9ad9-c7bd86d73700 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:55.739074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e480a3c7-21b6-4e76-9ad9-c7bd86d73700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:55.744523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 848/1712] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:55 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:45:55.754037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a632c-ff54-49af-a1fb-1f1a047d3752 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:55.793807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:55.960115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 865/1713] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:45:55.961201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480a3c7-21b6-4e76-9ad9-c7bd86d73700 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:55.966964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a632c-ff54-49af-a1fb-1f1a047d3752 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:55.981511 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480a3c7-21b6-4e76-9ad9-c7bd86d73700 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:55.984074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d3095-9ec5-4b8b-8090-73a1599607fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:55.986289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d3095-9ec5-4b8b-8090-73a1599607fa admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:55.986588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d3095-9ec5-4b8b-8090-73a1599607fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:55.986867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d3095-9ec5-4b8b-8090-73a1599607fa admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:55.989015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d3095-9ec5-4b8b-8090-73a1599607fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:55.994243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d3095-9ec5-4b8b-8090-73a1599607fa admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:56.000853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b85da-0ac6-4853-8256-6e1e7707d81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:56.001864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b85da-0ac6-4853-8256-6e1e7707d81f admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:56.001864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b85da-0ac6-4853-8256-6e1e7707d81f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:56.002057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b85da-0ac6-4853-8256-6e1e7707d81f admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:56.008743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b85da-0ac6-4853-8256-6e1e7707d81f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:45:56.009066 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b85da-0ac6-4853-8256-6e1e7707d81f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:56.017719 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b85da-0ac6-4853-8256-6e1e7707d81f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:56.045828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 866/1714] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:45:56.047749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b85da-0ac6-4853-8256-6e1e7707d81f admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:56.076894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 849/1715] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:55 2022] POST /identity/v3/users => generated 289 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:56.081711 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d3095-9ec5-4b8b-8090-73a1599607fa admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:56.087402 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b85da-0ac6-4853-8256-6e1e7707d81f admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:56.116442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d3095-9ec5-4b8b-8090-73a1599607fa admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:56.126978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91363d1d-4609-4668-9f43-4c789684d370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:56.130144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91363d1d-4609-4668-9f43-4c789684d370 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:56.130882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91363d1d-4609-4668-9f43-4c789684d370 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:56.131154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91363d1d-4609-4668-9f43-4c789684d370 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:56.133594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91363d1d-4609-4668-9f43-4c789684d370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:56.134433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff7818-dc37-4558-98fe-427652779ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:56.135522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff7818-dc37-4558-98fe-427652779ebe admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:56.135522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff7818-dc37-4558-98fe-427652779ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:56.135778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff7818-dc37-4558-98fe-427652779ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:56.136435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff7818-dc37-4558-98fe-427652779ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:56.139155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff7818-dc37-4558-98fe-427652779ebe admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:56.140708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91363d1d-4609-4668-9f43-4c789684d370 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:56.176174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 850/1716] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:56.181578 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91363d1d-4609-4668-9f43-4c789684d370 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:56.198186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91363d1d-4609-4668-9f43-4c789684d370 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:56.203802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff69fa4-3021-472f-a2a0-6c2080a9c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:56.204921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff69fa4-3021-472f-a2a0-6c2080a9c225 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:56.205723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff69fa4-3021-472f-a2a0-6c2080a9c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:56.205723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff69fa4-3021-472f-a2a0-6c2080a9c225 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:56.206377 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff69fa4-3021-472f-a2a0-6c2080a9c225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:56.209403 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff69fa4-3021-472f-a2a0-6c2080a9c225 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:56.216299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 867/1717] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:56 2022] POST /identity/v3/projects => generated 363 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:45:56.220396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff7818-dc37-4558-98fe-427652779ebe admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:56.249964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 851/1718] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:56.251253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff69fa4-3021-472f-a2a0-6c2080a9c225 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:56.299309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff69fa4-3021-472f-a2a0-6c2080a9c225 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:56.362279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff7818-dc37-4558-98fe-427652779ebe admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:56.372085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a62ad6-44c5-4580-bfde-1fa7598d871e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:56.373156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a62ad6-44c5-4580-bfde-1fa7598d871e admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:56.373668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a62ad6-44c5-4580-bfde-1fa7598d871e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:56.374174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a62ad6-44c5-4580-bfde-1fa7598d871e admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:56.375057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a62ad6-44c5-4580-bfde-1fa7598d871e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:56.379794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a62ad6-44c5-4580-bfde-1fa7598d871e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:56.422926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 868/1719] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:56 2022] POST /identity/v3/users => generated 282 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:56.427382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a62ad6-44c5-4580-bfde-1fa7598d871e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:56.455319 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a62ad6-44c5-4580-bfde-1fa7598d871e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:56.483702 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae79ec-38c8-4d71-a69a-c715cd81ba8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:56.487004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae79ec-38c8-4d71-a69a-c715cd81ba8c admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:56.488954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae79ec-38c8-4d71-a69a-c715cd81ba8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:56.490215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae79ec-38c8-4d71-a69a-c715cd81ba8c admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:56.491159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae79ec-38c8-4d71-a69a-c715cd81ba8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:56.493864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6dafd-799a-434f-8d14-a0113846f14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:56.494624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6dafd-799a-434f-8d14-a0113846f14d admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:56.494910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6dafd-799a-434f-8d14-a0113846f14d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:56.495245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6dafd-799a-434f-8d14-a0113846f14d admin admin] PATH_INFO: `/v3/projects/a23ad8e74b2e4740aaa6437312d0ee61/users/0c4b6c7c115e4050aa879477878a0e5c/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:56.499724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae79ec-38c8-4d71-a69a-c715cd81ba8c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:56.508239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:56.535823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 869/1720] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:56.537221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae79ec-38c8-4d71-a69a-c715cd81ba8c admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:56.539965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6dafd-799a-434f-8d14-a0113846f14d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23ad8e74b2e4740aaa6437312d0ee61, user_id=0c4b6c7c115e4050aa879477878a0e5c, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:56.547629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6dafd-799a-434f-8d14-a0113846f14d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:56.557191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae79ec-38c8-4d71-a69a-c715cd81ba8c admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:56.564412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859e9d9-0114-4738-ad5e-c90375e6e2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:56.565168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859e9d9-0114-4738-ad5e-c90375e6e2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:56.565567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859e9d9-0114-4738-ad5e-c90375e6e2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:56.566346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859e9d9-0114-4738-ad5e-c90375e6e2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:56.567556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859e9d9-0114-4738-ad5e-c90375e6e2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:56.570683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859e9d9-0114-4738-ad5e-c90375e6e2c2 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:56.587179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 852/1721] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:56 2022] PUT /identity/v3/projects/a23ad8e74b2e4740aaa6437312d0ee61/users/0c4b6c7c115e4050aa879477878a0e5c/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:56.602028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1969341-2798-46e8-9e52-097f2254b89a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:56.603044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1969341-2798-46e8-9e52-097f2254b89a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:56.605756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1969341-2798-46e8-9e52-097f2254b89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:56.614207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 870/1722] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:56.620954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859e9d9-0114-4738-ad5e-c90375e6e2c2 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:56.673201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859e9d9-0114-4738-ad5e-c90375e6e2c2 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:56.737504 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-b1969341-2798-46e8-9e52-097f2254b89a None None] MFA Rules not processed for user `0c4b6c7c115e4050aa879477878a0e5c`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:56.740151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b1969341-2798-46e8-9e52-097f2254b89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:56.842360 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f03b4-ce0d-426e-98e5-8c7a9b7345ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:56.843242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f03b4-ce0d-426e-98e5-8c7a9b7345ef admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:56.845836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f03b4-ce0d-426e-98e5-8c7a9b7345ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:56.846203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f03b4-ce0d-426e-98e5-8c7a9b7345ef admin admin] PATH_INFO: `/v3/projects/f77ecf1ba9334648bf3b824f68086208/users/2d59a444a5dc4dac94454505b69f2efa/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:56.911771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f03b4-ce0d-426e-98e5-8c7a9b7345ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77ecf1ba9334648bf3b824f68086208, user_id=2d59a444a5dc4dac94454505b69f2efa, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:56.917229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f03b4-ce0d-426e-98e5-8c7a9b7345ef admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:56.919793 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:56.943465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 853/1723] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:56.952700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 871/1724] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:56 2022] PUT /identity/v3/projects/f77ecf1ba9334648bf3b824f68086208/users/2d59a444a5dc4dac94454505b69f2efa/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:56.963534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8df74d-bdda-49be-93d7-d42f50c1e1aa None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:56.963969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8df74d-bdda-49be-93d7-d42f50c1e1aa None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:56.964343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8df74d-bdda-49be-93d7-d42f50c1e1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:56.968532 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f03b4-ce0d-426e-98e5-8c7a9b7345ef admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:56.972299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e82f03b4-ce0d-426e-98e5-8c7a9b7345ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:57.062087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-4e8df74d-bdda-49be-93d7-d42f50c1e1aa None None] MFA Rules not processed for user `2d59a444a5dc4dac94454505b69f2efa`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:57.067073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4e8df74d-bdda-49be-93d7-d42f50c1e1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:57.275978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f03b4-ce0d-426e-98e5-8c7a9b7345ef admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:57.306062 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b800ad-2dbd-4392-b070-b5348d6e8e2a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '913a866ddc074b7f9aa142c70da98a69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:57.306873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b800ad-2dbd-4392-b070-b5348d6e8e2a service glance] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:57.307413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b800ad-2dbd-4392-b070-b5348d6e8e2a service glance] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:57.307834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b800ad-2dbd-4392-b070-b5348d6e8e2a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:57.332584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 854/1725] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:57.336344 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf7f95-6fcf-4296-8144-31511873dd3c None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:57.339107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf7f95-6fcf-4296-8144-31511873dd3c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:57.339586 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf7f95-6fcf-4296-8144-31511873dd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:57.437238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d0bf7f95-6fcf-4296-8144-31511873dd3c None None] MFA Rules not processed for user `acf57eb971eb4f78b173a5ab47160bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:57.439058 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d0bf7f95-6fcf-4296-8144-31511873dd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:57.535255 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b800ad-2dbd-4392-b070-b5348d6e8e2a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:57.537173 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b800ad-2dbd-4392-b070-b5348d6e8e2a service glance] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:57.575842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 872/1726] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:45:57.585762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b800ad-2dbd-4392-b070-b5348d6e8e2a service glance] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:57.598603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:57.718044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 855/1727] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:57.729007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b27be1-863b-4303-a43b-1f9eeeb759fe None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:57.729305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b27be1-863b-4303-a43b-1f9eeeb759fe None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:57.729590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b27be1-863b-4303-a43b-1f9eeeb759fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:57.823085 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-b8b27be1-863b-4303-a43b-1f9eeeb759fe None None] MFA Rules not processed for user `acf57eb971eb4f78b173a5ab47160bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:57.824344 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b8b27be1-863b-4303-a43b-1f9eeeb759fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:57.869736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 856/1728] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:45:57.890597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b27be1-863b-4303-a43b-1f9eeeb759fe None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:57.892075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b8b27be1-863b-4303-a43b-1f9eeeb759fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:57.919551 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b800ad-2dbd-4392-b070-b5348d6e8e2a service glance] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:57.945923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad4494-b37b-451c-8f02-fb602a9a70f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:57.946639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad4494-b37b-451c-8f02-fb602a9a70f0 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:57.946919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad4494-b37b-451c-8f02-fb602a9a70f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:57.947224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad4494-b37b-451c-8f02-fb602a9a70f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:57.947826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad4494-b37b-451c-8f02-fb602a9a70f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:57.949799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad4494-b37b-451c-8f02-fb602a9a70f0 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:58.037353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 873/1729] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:45:57 2022] POST /identity/v3/projects => generated 363 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:58.038377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f1ad3-d5e0-47a2-b864-88ce8bc21c08 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:58.038668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f1ad3-d5e0-47a2-b864-88ce8bc21c08 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:58.038959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f1ad3-d5e0-47a2-b864-88ce8bc21c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:58.086904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b27be1-863b-4303-a43b-1f9eeeb759fe None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:58.102054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:58.186783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-767f1ad3-d5e0-47a2-b864-88ce8bc21c08 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:58.188190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-767f1ad3-d5e0-47a2-b864-88ce8bc21c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:58.324304 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed33cd2-f6a5-43d7-b2d2-13adefc2b322 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:58.326030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed33cd2-f6a5-43d7-b2d2-13adefc2b322 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:58.326442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed33cd2-f6a5-43d7-b2d2-13adefc2b322 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:58.327880 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed33cd2-f6a5-43d7-b2d2-13adefc2b322 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:58.347338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 874/1730] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:45:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:58.348285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f1ad3-d5e0-47a2-b864-88ce8bc21c08 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:58.558550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f1ad3-d5e0-47a2-b864-88ce8bc21c08 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:58.575023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed33cd2-f6a5-43d7-b2d2-13adefc2b322 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:58.577344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d21a37-2b50-4928-b29b-84d7d3256106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:58.578524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed33cd2-f6a5-43d7-b2d2-13adefc2b322 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:58.580543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d21a37-2b50-4928-b29b-84d7d3256106 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:58.580844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d21a37-2b50-4928-b29b-84d7d3256106 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:58.581531 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d21a37-2b50-4928-b29b-84d7d3256106 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:58.585053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d21a37-2b50-4928-b29b-84d7d3256106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:58.591724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d21a37-2b50-4928-b29b-84d7d3256106 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:58.606248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 857/1731] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:45:58.607565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66928ee-efdd-4538-90ed-a72bcd717ce1 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:58.607854 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66928ee-efdd-4538-90ed-a72bcd717ce1 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:58.608135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66928ee-efdd-4538-90ed-a72bcd717ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:58.630238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:58.678682 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 875/1732] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:45:58 2022] POST /identity/v3/users => generated 288 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:45:58.679659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d21a37-2b50-4928-b29b-84d7d3256106 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:58.687652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e9d21a37-2b50-4928-b29b-84d7d3256106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:58.688139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d66928ee-efdd-4538-90ed-a72bcd717ce1 None None] MFA Rules not processed for user `ed4b7bfb87b144b38b82514c17a468bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:58.690252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d66928ee-efdd-4538-90ed-a72bcd717ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:58.772634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d21a37-2b50-4928-b29b-84d7d3256106 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:58.800419 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:58.801079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:58.801368 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:58.801647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] PATH_INFO: `/v3/users/973b2f1bc42a48a7bc488889d5565944` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:58.805056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=973b2f1bc42a48a7bc488889d5565944)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:58.818060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:58.873037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 858/1733] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:58.873037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66928ee-efdd-4538-90ed-a72bcd717ce1 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:58.874587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d66928ee-efdd-4538-90ed-a72bcd717ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:59.027806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '973b2f1bc42a48a7bc488889d5565944'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:59.031023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '973b2f1bc42a48a7bc488889d5565944'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:59.044291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:59.070093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '973b2f1bc42a48a7bc488889d5565944'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:59.072961 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '973b2f1bc42a48a7bc488889d5565944'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:59.119792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '973b2f1bc42a48a7bc488889d5565944'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:45:59.142606 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 876/1734] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:45:58 2022] DELETE /identity/v3/users/973b2f1bc42a48a7bc488889d5565944 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:59.143916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:59.148712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66928ee-efdd-4538-90ed-a72bcd717ce1 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:59.149746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:59.338121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b25f0d-d645-4c12-b73c-0b0804508f15 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:59.356124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426d686-187b-4573-835c-3afe7ca3dd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:59.356849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426d686-187b-4573-835c-3afe7ca3dd7c admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:59.357020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426d686-187b-4573-835c-3afe7ca3dd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:59.357289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426d686-187b-4573-835c-3afe7ca3dd7c admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:59.362077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426d686-187b-4573-835c-3afe7ca3dd7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:59.364370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426d686-187b-4573-835c-3afe7ca3dd7c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:59.378744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7f69d-92ff-4df5-bb94-07642ba24395 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:59.379491 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7f69d-92ff-4df5-bb94-07642ba24395 service swift] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:59.379766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7f69d-92ff-4df5-bb94-07642ba24395 service swift] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:59.380097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7f69d-92ff-4df5-bb94-07642ba24395 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:59.381981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1dc7f69d-92ff-4df5-bb94-07642ba24395 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:59.389573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 877/1735] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:59.406365 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b14b93-b40a-4521-9007-5dbc61887af8 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:59.406883 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b14b93-b40a-4521-9007-5dbc61887af8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:59.407300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b14b93-b40a-4521-9007-5dbc61887af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:59.434525 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-17b14b93-b40a-4521-9007-5dbc61887af8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:45:59.436086 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-17b14b93-b40a-4521-9007-5dbc61887af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:59.450435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 878/1736] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:45:59.451353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b14b93-b40a-4521-9007-5dbc61887af8 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:59.465942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b14b93-b40a-4521-9007-5dbc61887af8 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:59.471920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3c16e-fd20-4429-afc2-291da59dfe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:59.472610 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3c16e-fd20-4429-afc2-291da59dfe0c admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:59.473383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3c16e-fd20-4429-afc2-291da59dfe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:59.473676 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3c16e-fd20-4429-afc2-291da59dfe0c admin admin] PATH_INFO: `/v3/projects/bb7a1305a54746f3b0f040a45df8f95d/users/fd53100f5076423886a1083125d14dde/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:59.500791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7f69d-92ff-4df5-bb94-07642ba24395 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:59.502661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7f69d-92ff-4df5-bb94-07642ba24395 service swift] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:59.513439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 859/1737] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:45:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:45:59.516730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3c16e-fd20-4429-afc2-291da59dfe0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb7a1305a54746f3b0f040a45df8f95d, user_id=fd53100f5076423886a1083125d14dde, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:59.517012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7f69d-92ff-4df5-bb94-07642ba24395 service swift] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:59.523826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1dc7f69d-92ff-4df5-bb94-07642ba24395 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:45:59.535284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3c16e-fd20-4429-afc2-291da59dfe0c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:59.594353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 879/1738] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:45:59 2022] PUT /identity/v3/projects/bb7a1305a54746f3b0f040a45df8f95d/users/fd53100f5076423886a1083125d14dde/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:45:59.601205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3c16e-fd20-4429-afc2-291da59dfe0c admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:45:59.672495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7f69d-92ff-4df5-bb94-07642ba24395 service swift] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:59.769939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:59.785721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:45:59.832154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cdef7-5d58-43a3-95f9-1d3499b9e13f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:59.833087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cdef7-5d58-43a3-95f9-1d3499b9e13f service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:59.833544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cdef7-5d58-43a3-95f9-1d3499b9e13f service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:59.834023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cdef7-5d58-43a3-95f9-1d3499b9e13f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:59.920915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3c16e-fd20-4429-afc2-291da59dfe0c admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:45:59.944434 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6052202-ff6d-4da1-a706-790a38a72d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:45:59.945322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6052202-ff6d-4da1-a706-790a38a72d7b admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:59.945322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6052202-ff6d-4da1-a706-790a38a72d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:59.945762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6052202-ff6d-4da1-a706-790a38a72d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:45:59.946856 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6052202-ff6d-4da1-a706-790a38a72d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:45:59.948524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6052202-ff6d-4da1-a706-790a38a72d7b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:45:59.974847 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 880/1739] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:45:59.976095 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fedc6-a4e1-4969-9990-a3673b0a9253 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:45:59.977874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fedc6-a4e1-4969-9990-a3673b0a9253 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:45:59.977874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fedc6-a4e1-4969-9990-a3673b0a9253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:00.032566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-548fedc6-a4e1-4969-9990-a3673b0a9253 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:00.034296 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-548fedc6-a4e1-4969-9990-a3673b0a9253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:00.067520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cdef7-5d58-43a3-95f9-1d3499b9e13f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:00.071800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cdef7-5d58-43a3-95f9-1d3499b9e13f service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:00.102081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 860/1740] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:46:00.102081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10c1d2-91d4-4463-ad54-93591a9055d8 None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:00.102081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10c1d2-91d4-4463-ad54-93591a9055d8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:00.102081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10c1d2-91d4-4463-ad54-93591a9055d8 None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:00.102081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 861/1741] 10.222.0.26 () {54 vars in 912 bytes} [Tue Oct 4 08:46:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:00.106251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10c1d2-91d4-4463-ad54-93591a9055d8 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:00.144528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10c1d2-91d4-4463-ad54-93591a9055d8 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:00.151127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f1844-be04-4010-af45-8e61cae84407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:00.152049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f1844-be04-4010-af45-8e61cae84407 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:00.152395 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f1844-be04-4010-af45-8e61cae84407 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:00.152736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f1844-be04-4010-af45-8e61cae84407 admin admin] PATH_INFO: `/v3/projects/bb7a1305a54746f3b0f040a45df8f95d/users/fd53100f5076423886a1083125d14dde/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:00.153347 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 881/1742] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:45:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:00.155496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2360aed-4b52-423c-ae44-e90b4c2f19a7 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:00.155970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2360aed-4b52-423c-ae44-e90b4c2f19a7 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:00.156580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2360aed-4b52-423c-ae44-e90b4c2f19a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:00.169982 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f1844-be04-4010-af45-8e61cae84407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb7a1305a54746f3b0f040a45df8f95d, user_id=fd53100f5076423886a1083125d14dde, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:00.177163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f1844-be04-4010-af45-8e61cae84407 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:00.206151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 862/1743] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:00 2022] PUT /identity/v3/projects/bb7a1305a54746f3b0f040a45df8f95d/users/fd53100f5076423886a1083125d14dde/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:00.206151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f1844-be04-4010-af45-8e61cae84407 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:00.206151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b77f1844-be04-4010-af45-8e61cae84407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:00.226918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-c2360aed-4b52-423c-ae44-e90b4c2f19a7 None None] MFA Rules not processed for user `50d3b75cbc5c4d1686f4d93248e04746`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:00.234712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c2360aed-4b52-423c-ae44-e90b4c2f19a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:00.416836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 882/1744] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:46:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:00.418216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237800ce-94b4-4f5e-bf63-8bfd86926b2c None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:00.418522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237800ce-94b4-4f5e-bf63-8bfd86926b2c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:00.418842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237800ce-94b4-4f5e-bf63-8bfd86926b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:00.490000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f1844-be04-4010-af45-8e61cae84407 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:00.507256 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756ad9d-d064-4c03-9498-ad3634359256 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:00.507959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756ad9d-d064-4c03-9498-ad3634359256 service swift] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:00.508231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756ad9d-d064-4c03-9498-ad3634359256 service swift] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:00.508543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756ad9d-d064-4c03-9498-ad3634359256 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:00.512013 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-237800ce-94b4-4f5e-bf63-8bfd86926b2c None None] MFA Rules not processed for user `fd53100f5076423886a1083125d14dde`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:00.513331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-237800ce-94b4-4f5e-bf63-8bfd86926b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:00.644366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756ad9d-d064-4c03-9498-ad3634359256 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:00.646021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756ad9d-d064-4c03-9498-ad3634359256 service swift] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:00.669434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 863/1745] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:46:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:46:00.670367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756ad9d-d064-4c03-9498-ad3634359256 service swift] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:00.688557 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 883/1746] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:00.692196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756ad9d-d064-4c03-9498-ad3634359256 service swift] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:00.701801 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237800ce-94b4-4f5e-bf63-8bfd86926b2c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:00.703415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-237800ce-94b4-4f5e-bf63-8bfd86926b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:00.707756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd428784-fe42-465d-87c0-76dc386378f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:00.710028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd428784-fe42-465d-87c0-76dc386378f6 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:00.710313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd428784-fe42-465d-87c0-76dc386378f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:00.710587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd428784-fe42-465d-87c0-76dc386378f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:00.715165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:00.737328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:00.743245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd428784-fe42-465d-87c0-76dc386378f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:00.758122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd428784-fe42-465d-87c0-76dc386378f6 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:00.771221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 864/1747] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:00.826878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237800ce-94b4-4f5e-bf63-8bfd86926b2c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:00.850538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91a76f-e605-4fc2-8f4e-fe952d493f9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:00.852861 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91a76f-e605-4fc2-8f4e-fe952d493f9b service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:00.852861 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91a76f-e605-4fc2-8f4e-fe952d493f9b service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:00.853561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91a76f-e605-4fc2-8f4e-fe952d493f9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:00.890959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91a76f-e605-4fc2-8f4e-fe952d493f9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:00.892494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91a76f-e605-4fc2-8f4e-fe952d493f9b service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:00.917882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 884/1748] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:46:00.961197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd428784-fe42-465d-87c0-76dc386378f6 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:00.963967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-fd428784-fe42-465d-87c0-76dc386378f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:01.265595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91a76f-e605-4fc2-8f4e-fe952d493f9b service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:01.267883 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2c91a76f-e605-4fc2-8f4e-fe952d493f9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:01.297617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91a76f-e605-4fc2-8f4e-fe952d493f9b service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:01.314530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4366a4-4181-4595-b834-2cc5e20fad19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:01.315822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4366a4-4181-4595-b834-2cc5e20fad19 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:01.319089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4366a4-4181-4595-b834-2cc5e20fad19 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:01.320172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4366a4-4181-4595-b834-2cc5e20fad19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:01.330278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-bb4366a4-4181-4595-b834-2cc5e20fad19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:01.335974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd428784-fe42-465d-87c0-76dc386378f6 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:01.361058 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:01.364604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:01.364900 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:01.368776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d admin admin] PATH_INFO: `/v3/projects/281ccf5c1a784a89b9f4f5e0905f33ca` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:01.374780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=281ccf5c1a784a89b9f4f5e0905f33ca)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:01.383459 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:01.575707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4366a4-4181-4595-b834-2cc5e20fad19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:01.581327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4366a4-4181-4595-b834-2cc5e20fad19 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:01.618733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 885/1749] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:46:01.622264 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '281ccf5c1a784a89b9f4f5e0905f33ca', 'request_id': 'req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:01.658357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '281ccf5c1a784a89b9f4f5e0905f33ca', 'request_id': 'req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:01.697449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '281ccf5c1a784a89b9f4f5e0905f33ca', 'request_id': 'req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:01.750391 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d admin admin] The token cache is being invalidate because project 281ccf5c1a784a89b9f4f5e0905f33ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:01.750760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:01.755666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8216a46d-0ff8-4bef-9ca9-306e8072ca6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:01.761150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 865/1750] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:00 2022] DELETE /identity/v3/projects/281ccf5c1a784a89b9f4f5e0905f33ca => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:01.766148 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c258b7-8008-493d-9c86-82a59f32facc None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:01.766448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c258b7-8008-493d-9c86-82a59f32facc None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:01.766731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c258b7-8008-493d-9c86-82a59f32facc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:01.841636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-51c258b7-8008-493d-9c86-82a59f32facc None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:01.854006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-51c258b7-8008-493d-9c86-82a59f32facc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:02.048390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 886/1751] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:02.057973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64525b2a-a709-4f29-96af-56c884ba13a3 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:02.058264 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64525b2a-a709-4f29-96af-56c884ba13a3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:02.058546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64525b2a-a709-4f29-96af-56c884ba13a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:02.108433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-64525b2a-a709-4f29-96af-56c884ba13a3 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:02.109407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-64525b2a-a709-4f29-96af-56c884ba13a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:02.143913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 866/1752] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:02.148600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c258b7-8008-493d-9c86-82a59f32facc None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:02.172091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c258b7-8008-493d-9c86-82a59f32facc None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:02.208195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb35cd6-6ce1-4dee-a670-a46667fdab00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:02.208407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb35cd6-6ce1-4dee-a670-a46667fdab00 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:02.208701 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb35cd6-6ce1-4dee-a670-a46667fdab00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:02.208978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb35cd6-6ce1-4dee-a670-a46667fdab00 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:02.209645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb35cd6-6ce1-4dee-a670-a46667fdab00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:02.209981 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb35cd6-6ce1-4dee-a670-a46667fdab00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:02.226013 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb35cd6-6ce1-4dee-a670-a46667fdab00 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:02.264770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 887/1753] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:46:02.266025 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64525b2a-a709-4f29-96af-56c884ba13a3 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:02.276682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64525b2a-a709-4f29-96af-56c884ba13a3 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:02.303714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b11fa-99c1-48f6-9c24-7ea34e1403ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:02.304517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b11fa-99c1-48f6-9c24-7ea34e1403ab admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:02.306125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b11fa-99c1-48f6-9c24-7ea34e1403ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:02.306877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b11fa-99c1-48f6-9c24-7ea34e1403ab admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:02.307768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b11fa-99c1-48f6-9c24-7ea34e1403ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:02.314772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442b11fa-99c1-48f6-9c24-7ea34e1403ab admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:02.383366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 867/1754] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:02 2022] POST /identity/v3/projects => generated 355 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:02.387884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb35cd6-6ce1-4dee-a670-a46667fdab00 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:02.403381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb35cd6-6ce1-4dee-a670-a46667fdab00 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:02.413301 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38d596-e693-4760-9b90-d2ad737269b7 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:02.413593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38d596-e693-4760-9b90-d2ad737269b7 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:02.413902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38d596-e693-4760-9b90-d2ad737269b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:02.460234 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-3e38d596-e693-4760-9b90-d2ad737269b7 None None] MFA Rules not processed for user `3c0b7528b0ff4b09923a0e951d052ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:02.462942 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3e38d596-e693-4760-9b90-d2ad737269b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:02.498998 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:02.566386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924ed2-e0a4-4307-a1c5-809839c42c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:02.567856 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924ed2-e0a4-4307-a1c5-809839c42c1b admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:02.568163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924ed2-e0a4-4307-a1c5-809839c42c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:02.568459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924ed2-e0a4-4307-a1c5-809839c42c1b admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:02.569146 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f924ed2-e0a4-4307-a1c5-809839c42c1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:02.571190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f924ed2-e0a4-4307-a1c5-809839c42c1b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:02.633020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 888/1755] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:02 2022] POST /identity/v3/users => generated 278 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:46:02.636829 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924ed2-e0a4-4307-a1c5-809839c42c1b admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:02.649961 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 868/1756] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:02.665861 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924ed2-e0a4-4307-a1c5-809839c42c1b admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:02.702638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c927b-8dae-4a86-bb51-19e0e7a1d146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:02.703276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c927b-8dae-4a86-bb51-19e0e7a1d146 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:02.703558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c927b-8dae-4a86-bb51-19e0e7a1d146 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:02.703836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c927b-8dae-4a86-bb51-19e0e7a1d146 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:02.704501 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c927b-8dae-4a86-bb51-19e0e7a1d146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:02.706146 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c927b-8dae-4a86-bb51-19e0e7a1d146 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:02.732860 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 889/1757] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:02.736808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38d596-e693-4760-9b90-d2ad737269b7 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:02.749340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:02.766316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dc6e6-6a22-4a05-b8c6-a9c6f0a5a161 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:02.766678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dc6e6-6a22-4a05-b8c6-a9c6f0a5a161 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:02.766906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38dc6e6-6a22-4a05-b8c6-a9c6f0a5a161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:02.768839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38d596-e693-4760-9b90-d2ad737269b7 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:02.782895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2a7fb-2a3a-4993-aa9e-6161588dac29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:02.783596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2a7fb-2a3a-4993-aa9e-6161588dac29 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:02.783885 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2a7fb-2a3a-4993-aa9e-6161588dac29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:02.788093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2a7fb-2a3a-4993-aa9e-6161588dac29 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:02.788812 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2a7fb-2a3a-4993-aa9e-6161588dac29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:02.796217 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2a7fb-2a3a-4993-aa9e-6161588dac29 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:02.808498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-e38dc6e6-6a22-4a05-b8c6-a9c6f0a5a161 None None] MFA Rules not processed for user `acf57eb971eb4f78b173a5ab47160bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:02.809790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e38dc6e6-6a22-4a05-b8c6-a9c6f0a5a161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:02.814190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 869/1758] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:02.817111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2a7fb-2a3a-4993-aa9e-6161588dac29 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:02.830777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2a7fb-2a3a-4993-aa9e-6161588dac29 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:02.846712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221db08e-37d7-430e-beb8-73aeccf0abde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:02.847966 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221db08e-37d7-430e-beb8-73aeccf0abde admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:02.848255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221db08e-37d7-430e-beb8-73aeccf0abde admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:02.848535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221db08e-37d7-430e-beb8-73aeccf0abde admin admin] PATH_INFO: `/v3/projects/8f6bca0f4f83430d98e51062f19d875f/users/25febc0733ac41edbecde156f7cb8f46/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:02.879357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221db08e-37d7-430e-beb8-73aeccf0abde admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6bca0f4f83430d98e51062f19d875f, user_id=25febc0733ac41edbecde156f7cb8f46, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:02.883170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221db08e-37d7-430e-beb8-73aeccf0abde admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:02.906841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 870/1759] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:02 2022] PUT /identity/v3/projects/8f6bca0f4f83430d98e51062f19d875f/users/25febc0733ac41edbecde156f7cb8f46/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:02.918730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828f4e-7ad7-4274-877e-d2d0254d3483 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:02.919178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828f4e-7ad7-4274-877e-d2d0254d3483 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:02.919955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db828f4e-7ad7-4274-877e-d2d0254d3483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:02.961218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-db828f4e-7ad7-4274-877e-d2d0254d3483 None None] MFA Rules not processed for user `25febc0733ac41edbecde156f7cb8f46`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:02.964085 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-db828f4e-7ad7-4274-877e-d2d0254d3483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:02.979581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 890/1760] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:03.056073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 871/1761] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:03.060471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dc6e6-6a22-4a05-b8c6-a9c6f0a5a161 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:03.075012 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38dc6e6-6a22-4a05-b8c6-a9c6f0a5a161 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:03.180792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcf956-edc3-4916-b2c3-6379f81ee7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:03.181557 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf956-edc3-4916-b2c3-6379f81ee7cb admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:03.181839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf956-edc3-4916-b2c3-6379f81ee7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:03.182126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf956-edc3-4916-b2c3-6379f81ee7cb admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:03.182786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcf956-edc3-4916-b2c3-6379f81ee7cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:03.187086 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcf956-edc3-4916-b2c3-6379f81ee7cb admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:03.232462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 891/1762] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:03 2022] POST /identity/v3/projects => generated 357 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:03.236675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db828f4e-7ad7-4274-877e-d2d0254d3483 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:03.250115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db828f4e-7ad7-4274-877e-d2d0254d3483 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:03.385678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61535e-a36e-4ecc-8204-e7e7485a2059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:03.386365 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61535e-a36e-4ecc-8204-e7e7485a2059 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:03.386756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61535e-a36e-4ecc-8204-e7e7485a2059 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:03.387156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61535e-a36e-4ecc-8204-e7e7485a2059 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:03.387942 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61535e-a36e-4ecc-8204-e7e7485a2059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:03.390105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61535e-a36e-4ecc-8204-e7e7485a2059 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:03.423863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 872/1763] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:03 2022] POST /identity/v3/users => generated 285 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:46:03.428182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcf956-edc3-4916-b2c3-6379f81ee7cb admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:03.440345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcf956-edc3-4916-b2c3-6379f81ee7cb admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:03.445610 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f852e-32ea-4873-a054-f5b3c972de5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:03.446808 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f852e-32ea-4873-a054-f5b3c972de5c admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:03.447092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f852e-32ea-4873-a054-f5b3c972de5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:03.447369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f852e-32ea-4873-a054-f5b3c972de5c admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:03.448019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f852e-32ea-4873-a054-f5b3c972de5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:03.449751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f852e-32ea-4873-a054-f5b3c972de5c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:03.463324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 892/1764] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:03.467770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61535e-a36e-4ecc-8204-e7e7485a2059 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:03.480806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61535e-a36e-4ecc-8204-e7e7485a2059 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:03.489735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e8f1-0f73-433a-8e1d-d0aa4c422b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:03.490708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e8f1-0f73-433a-8e1d-d0aa4c422b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:03.491050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e8f1-0f73-433a-8e1d-d0aa4c422b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:03.491330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e8f1-0f73-433a-8e1d-d0aa4c422b7b admin admin] PATH_INFO: `/v3/projects/67f34368087a456291d2ad4d9aa5e1ef/users/66ff0624d9bc4ee98795e1d39757de8c/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:03.535123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350e8f1-0f73-433a-8e1d-d0aa4c422b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=67f34368087a456291d2ad4d9aa5e1ef, user_id=66ff0624d9bc4ee98795e1d39757de8c, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:03.538195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350e8f1-0f73-433a-8e1d-d0aa4c422b7b admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:03.555840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 873/1765] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:03 2022] PUT /identity/v3/projects/67f34368087a456291d2ad4d9aa5e1ef/users/66ff0624d9bc4ee98795e1d39757de8c/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:03.560436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f852e-32ea-4873-a054-f5b3c972de5c admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:03.575931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f852e-32ea-4873-a054-f5b3c972de5c admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:03.636804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:03.686690 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ff7f9-d6d4-4ce5-ad3c-9ddcda223225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:03.687482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ff7f9-d6d4-4ce5-ad3c-9ddcda223225 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:03.687764 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ff7f9-d6d4-4ce5-ad3c-9ddcda223225 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:03.688044 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ff7f9-d6d4-4ce5-ad3c-9ddcda223225 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:03.688699 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ff7f9-d6d4-4ce5-ad3c-9ddcda223225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:03.690375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ff7f9-d6d4-4ce5-ad3c-9ddcda223225 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:03.706374 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 893/1766] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:03.710855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e8f1-0f73-433a-8e1d-d0aa4c422b7b admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:03.722727 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e8f1-0f73-433a-8e1d-d0aa4c422b7b admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:03.731475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2953b-1245-4e1a-a9e1-3f1c68183163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:03.732165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2953b-1245-4e1a-a9e1-3f1c68183163 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:03.732446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2953b-1245-4e1a-a9e1-3f1c68183163 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:03.732720 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2953b-1245-4e1a-a9e1-3f1c68183163 admin admin] PATH_INFO: `/v3/projects/67f34368087a456291d2ad4d9aa5e1ef/users/66ff0624d9bc4ee98795e1d39757de8c/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:03.734706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2953b-1245-4e1a-a9e1-3f1c68183163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67f34368087a456291d2ad4d9aa5e1ef, user_id=66ff0624d9bc4ee98795e1d39757de8c, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:03.737436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2953b-1245-4e1a-a9e1-3f1c68183163 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:03.750287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 874/1767] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:03 2022] PUT /identity/v3/projects/67f34368087a456291d2ad4d9aa5e1ef/users/66ff0624d9bc4ee98795e1d39757de8c/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:03.762037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0a7ce-4dcf-489b-b534-4b0bdf4c49bc None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:03.762337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0a7ce-4dcf-489b-b534-4b0bdf4c49bc None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:03.762725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0a7ce-4dcf-489b-b534-4b0bdf4c49bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:03.794903 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-34a0a7ce-4dcf-489b-b534-4b0bdf4c49bc None None] MFA Rules not processed for user `66ff0624d9bc4ee98795e1d39757de8c`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:03.796148 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-34a0a7ce-4dcf-489b-b534-4b0bdf4c49bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:03.878605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 894/1768] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:03.890937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2953b-1245-4e1a-a9e1-3f1c68183163 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:03.892664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3ee2953b-1245-4e1a-a9e1-3f1c68183163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:03.905418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:03.989489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2953b-1245-4e1a-a9e1-3f1c68183163 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:03.994908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f91e9-db46-443d-b8bb-82d207d88b36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:03.996163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f91e9-db46-443d-b8bb-82d207d88b36 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:03.996526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f91e9-db46-443d-b8bb-82d207d88b36 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:03.996805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f91e9-db46-443d-b8bb-82d207d88b36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:04.024635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f91e9-db46-443d-b8bb-82d207d88b36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:04.026158 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f91e9-db46-443d-b8bb-82d207d88b36 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:04.038778 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 875/1769] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:04.155138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0a7ce-4dcf-489b-b534-4b0bdf4c49bc None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:04.167444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0a7ce-4dcf-489b-b534-4b0bdf4c49bc None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:04.182149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e620e9a-5f32-4a7f-8705-a2e0974d614e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:04.182774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e620e9a-5f32-4a7f-8705-a2e0974d614e service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:04.183054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e620e9a-5f32-4a7f-8705-a2e0974d614e service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:04.183323 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e620e9a-5f32-4a7f-8705-a2e0974d614e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:04.278481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e620e9a-5f32-4a7f-8705-a2e0974d614e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:04.279984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e620e9a-5f32-4a7f-8705-a2e0974d614e service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:04.309936 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 895/1770] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:07.146800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f91e9-db46-443d-b8bb-82d207d88b36 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:07.148257 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8b0f91e9-db46-443d-b8bb-82d207d88b36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:07.215611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f91e9-db46-443d-b8bb-82d207d88b36 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:07.221001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:07.221655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:07.222002 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:07.222273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] PATH_INFO: `/v3/users/2d59a444a5dc4dac94454505b69f2efa` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:07.223434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d59a444a5dc4dac94454505b69f2efa)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:07.225411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:07.294459 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d59a444a5dc4dac94454505b69f2efa'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:07.295566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d59a444a5dc4dac94454505b69f2efa'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:07.318389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d59a444a5dc4dac94454505b69f2efa'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:07.326999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d59a444a5dc4dac94454505b69f2efa'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:07.327839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d59a444a5dc4dac94454505b69f2efa'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:07.344453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 876/1771] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:07 2022] DELETE /identity/v3/users/2d59a444a5dc4dac94454505b69f2efa => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:07.349370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7986b09-077b-4d40-ad5d-626b3c93854d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:07.349973 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7986b09-077b-4d40-ad5d-626b3c93854d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:07.349973 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7986b09-077b-4d40-ad5d-626b3c93854d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:07.369922 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b7986b09-077b-4d40-ad5d-626b3c93854d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:07.371158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b7986b09-077b-4d40-ad5d-626b3c93854d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:07.434633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:07.454437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 896/1772] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:07.461528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:07.462988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:07.501043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:07.533625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06959f2e-fbf1-473a-bef6-3ba3daaf48c1 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:07.537472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8754af2-735b-4fdc-b45d-cb1b9c65a859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:07.538317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8754af2-735b-4fdc-b45d-cb1b9c65a859 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:07.538601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8754af2-735b-4fdc-b45d-cb1b9c65a859 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:07.538876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8754af2-735b-4fdc-b45d-cb1b9c65a859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:07.556274 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8754af2-735b-4fdc-b45d-cb1b9c65a859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:07.557878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8754af2-735b-4fdc-b45d-cb1b9c65a859 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:07.570706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 877/1773] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:08.203128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7986b09-077b-4d40-ad5d-626b3c93854d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:08.204679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b7986b09-077b-4d40-ad5d-626b3c93854d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:08.248079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7986b09-077b-4d40-ad5d-626b3c93854d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:08.253811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:08.254683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:08.254966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:08.255247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] PATH_INFO: `/v3/projects/f77ecf1ba9334648bf3b824f68086208` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:08.256405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f77ecf1ba9334648bf3b824f68086208)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:08.258600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:08.320631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f77ecf1ba9334648bf3b824f68086208', 'request_id': 'req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:08.327730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f77ecf1ba9334648bf3b824f68086208', 'request_id': 'req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:08.334455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f77ecf1ba9334648bf3b824f68086208', 'request_id': 'req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:08.366394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] The token cache is being invalidate because project f77ecf1ba9334648bf3b824f68086208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:08.367420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:08.368451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:08.373781 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 897/1774] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:08 2022] DELETE /identity/v3/projects/f77ecf1ba9334648bf3b824f68086208 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:08.374642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8754af2-735b-4fdc-b45d-cb1b9c65a859 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:08.390733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8754af2-735b-4fdc-b45d-cb1b9c65a859 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:08.515439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba1ae6-57e2-45ce-ba91-1ecd0e5a7eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:08.516341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba1ae6-57e2-45ce-ba91-1ecd0e5a7eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:08.516801 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba1ae6-57e2-45ce-ba91-1ecd0e5a7eea admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:08.517066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba1ae6-57e2-45ce-ba91-1ecd0e5a7eea admin admin] PATH_INFO: `/v3/users/fd53100f5076423886a1083125d14dde` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:08.518277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba1ae6-57e2-45ce-ba91-1ecd0e5a7eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd53100f5076423886a1083125d14dde)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:08.520245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba1ae6-57e2-45ce-ba91-1ecd0e5a7eea admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:08.582756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-08ba1ae6-57e2-45ce-ba91-1ecd0e5a7eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd53100f5076423886a1083125d14dde'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:08.583588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-08ba1ae6-57e2-45ce-ba91-1ecd0e5a7eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd53100f5076423886a1083125d14dde'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:08.601678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-08ba1ae6-57e2-45ce-ba91-1ecd0e5a7eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd53100f5076423886a1083125d14dde'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:08.610652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-08ba1ae6-57e2-45ce-ba91-1ecd0e5a7eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd53100f5076423886a1083125d14dde'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:08.611415 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-08ba1ae6-57e2-45ce-ba91-1ecd0e5a7eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd53100f5076423886a1083125d14dde'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:08.630268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 878/1775] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:08 2022] DELETE /identity/v3/users/fd53100f5076423886a1083125d14dde => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:08.919536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:08.921533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:08.937088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058c6a7-5d43-4bc6-8589-b57a5232fdf3 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:09.024974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a2219-bbe6-4783-bb19-42265634b307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:09.025613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a2219-bbe6-4783-bb19-42265634b307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:09.025923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a2219-bbe6-4783-bb19-42265634b307 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:09.026194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a2219-bbe6-4783-bb19-42265634b307 admin admin] PATH_INFO: `/v3/projects/bb7a1305a54746f3b0f040a45df8f95d` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:09.027346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a2219-bbe6-4783-bb19-42265634b307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb7a1305a54746f3b0f040a45df8f95d)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:09.029298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a2219-bbe6-4783-bb19-42265634b307 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:09.112345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b15a2219-bbe6-4783-bb19-42265634b307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb7a1305a54746f3b0f040a45df8f95d', 'request_id': 'req-b15a2219-bbe6-4783-bb19-42265634b307'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:09.119670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b15a2219-bbe6-4783-bb19-42265634b307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb7a1305a54746f3b0f040a45df8f95d', 'request_id': 'req-b15a2219-bbe6-4783-bb19-42265634b307'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:09.125728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b15a2219-bbe6-4783-bb19-42265634b307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb7a1305a54746f3b0f040a45df8f95d', 'request_id': 'req-b15a2219-bbe6-4783-bb19-42265634b307'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:09.160592 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b15a2219-bbe6-4783-bb19-42265634b307 admin admin] The token cache is being invalidate because project bb7a1305a54746f3b0f040a45df8f95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:09.161453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b15a2219-bbe6-4783-bb19-42265634b307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:09.162365 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b15a2219-bbe6-4783-bb19-42265634b307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:09.165008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 898/1776] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:08 2022] DELETE /identity/v3/projects/bb7a1305a54746f3b0f040a45df8f95d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:09.170076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aefe65-dc05-41e8-bf69-269810507897 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:09.170362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aefe65-dc05-41e8-bf69-269810507897 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:09.170636 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aefe65-dc05-41e8-bf69-269810507897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:09.191797 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a8aefe65-dc05-41e8-bf69-269810507897 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:09.193013 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a8aefe65-dc05-41e8-bf69-269810507897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:09.301563 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:09.322079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 879/1777] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:09.329865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6622d-2cb9-4736-b0ac-b75c2e9562a1 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:09.330182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6622d-2cb9-4736-b0ac-b75c2e9562a1 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:09.330459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6622d-2cb9-4736-b0ac-b75c2e9562a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:09.340366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:09.356100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-ddd6622d-2cb9-4736-b0ac-b75c2e9562a1 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:09.357706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ddd6622d-2cb9-4736-b0ac-b75c2e9562a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:09.367015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 899/1778] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:09.371848 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aefe65-dc05-41e8-bf69-269810507897 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:09.383874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aefe65-dc05-41e8-bf69-269810507897 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:09.390318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a92f0-3d34-47b3-a5c7-4c9281b3094c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:09.391018 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a92f0-3d34-47b3-a5c7-4c9281b3094c admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:09.391228 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a92f0-3d34-47b3-a5c7-4c9281b3094c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:09.391530 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a92f0-3d34-47b3-a5c7-4c9281b3094c admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:09.392152 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a92f0-3d34-47b3-a5c7-4c9281b3094c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:09.392457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a92f0-3d34-47b3-a5c7-4c9281b3094c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:09.394113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a92f0-3d34-47b3-a5c7-4c9281b3094c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:09.406666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 880/1779] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:46:09.411195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6622d-2cb9-4736-b0ac-b75c2e9562a1 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:09.421018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6622d-2cb9-4736-b0ac-b75c2e9562a1 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:09.426539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a8c13-b0a4-474c-ad4a-c210ef6c06d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:09.427170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a8c13-b0a4-474c-ad4a-c210ef6c06d9 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:09.427484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a8c13-b0a4-474c-ad4a-c210ef6c06d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:09.427893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a8c13-b0a4-474c-ad4a-c210ef6c06d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:09.428557 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a8c13-b0a4-474c-ad4a-c210ef6c06d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:09.431277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a8c13-b0a4-474c-ad4a-c210ef6c06d9 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:09.463145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 900/1780] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:09 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:09.467689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a92f0-3d34-47b3-a5c7-4c9281b3094c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:09.481493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a92f0-3d34-47b3-a5c7-4c9281b3094c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:09.559263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc7436-dc97-4fd3-b901-888f18eba809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:09.560748 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc7436-dc97-4fd3-b901-888f18eba809 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:09.560748 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc7436-dc97-4fd3-b901-888f18eba809 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:09.560992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc7436-dc97-4fd3-b901-888f18eba809 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:09.561638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc7436-dc97-4fd3-b901-888f18eba809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:09.563770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc7436-dc97-4fd3-b901-888f18eba809 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:09.596573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 881/1781] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:09 2022] POST /identity/v3/users => generated 285 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:46:09.598239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a8c13-b0a4-474c-ad4a-c210ef6c06d9 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:09.609528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a8c13-b0a4-474c-ad4a-c210ef6c06d9 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:09.614214 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28606dac-263c-41b4-b94a-31fe76c78920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:09.614813 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28606dac-263c-41b4-b94a-31fe76c78920 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:09.615091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28606dac-263c-41b4-b94a-31fe76c78920 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:09.615381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28606dac-263c-41b4-b94a-31fe76c78920 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:09.616015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28606dac-263c-41b4-b94a-31fe76c78920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:09.618332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28606dac-263c-41b4-b94a-31fe76c78920 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:09.630115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 901/1782] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:09.634816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc7436-dc97-4fd3-b901-888f18eba809 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:09.645337 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc7436-dc97-4fd3-b901-888f18eba809 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:09.671903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d9ea8-d1bc-42d1-84d1-7469800fb107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:09.673076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d9ea8-d1bc-42d1-84d1-7469800fb107 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:09.673076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d9ea8-d1bc-42d1-84d1-7469800fb107 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:09.673255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d9ea8-d1bc-42d1-84d1-7469800fb107 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:09.673866 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d9ea8-d1bc-42d1-84d1-7469800fb107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:09.675700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d9ea8-d1bc-42d1-84d1-7469800fb107 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:09.688848 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 882/1783] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:09.693005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28606dac-263c-41b4-b94a-31fe76c78920 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:09.705875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28606dac-263c-41b4-b94a-31fe76c78920 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:09.707745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fcd84-f3ac-4418-9f4e-edbb26790062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:09.708577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fcd84-f3ac-4418-9f4e-edbb26790062 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:09.708676 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fcd84-f3ac-4418-9f4e-edbb26790062 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:09.708970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fcd84-f3ac-4418-9f4e-edbb26790062 admin admin] PATH_INFO: `/v3/projects/6662669c78994181bfe4b5dd7f9b1ec9/users/93c6829ab8da4c15a76b16af47637bb0/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:09.725313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665fcd84-f3ac-4418-9f4e-edbb26790062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6662669c78994181bfe4b5dd7f9b1ec9, user_id=93c6829ab8da4c15a76b16af47637bb0, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:09.728592 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665fcd84-f3ac-4418-9f4e-edbb26790062 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:09.743780 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 902/1784] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:09 2022] PUT /identity/v3/projects/6662669c78994181bfe4b5dd7f9b1ec9/users/93c6829ab8da4c15a76b16af47637bb0/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:09.764896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86288ec-3fd4-49b4-b232-1a229348f7d5 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:09.765382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86288ec-3fd4-49b4-b232-1a229348f7d5 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:09.765965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86288ec-3fd4-49b4-b232-1a229348f7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:09.812502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f86288ec-3fd4-49b4-b232-1a229348f7d5 None None] MFA Rules not processed for user `93c6829ab8da4c15a76b16af47637bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:09.814589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f86288ec-3fd4-49b4-b232-1a229348f7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:09.911959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 883/1785] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:09.913472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fcd84-f3ac-4418-9f4e-edbb26790062 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:10.006684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fcd84-f3ac-4418-9f4e-edbb26790062 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:10.018326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff3bcb-9684-4504-96ea-2c2ff80c18c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:10.019440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff3bcb-9684-4504-96ea-2c2ff80c18c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:10.019725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff3bcb-9684-4504-96ea-2c2ff80c18c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:10.020000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff3bcb-9684-4504-96ea-2c2ff80c18c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:10.020727 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff3bcb-9684-4504-96ea-2c2ff80c18c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:10.023304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff3bcb-9684-4504-96ea-2c2ff80c18c9 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:10.065481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 903/1786] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:09 2022] POST /identity/v3/projects => generated 369 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:10.074784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86288ec-3fd4-49b4-b232-1a229348f7d5 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:10.087759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86288ec-3fd4-49b4-b232-1a229348f7d5 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:10.171029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:10.184694 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f4898-8ec4-42f4-8901-5e27ad2546fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:10.186411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f4898-8ec4-42f4-8901-5e27ad2546fc admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:10.186871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f4898-8ec4-42f4-8901-5e27ad2546fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:10.187338 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f4898-8ec4-42f4-8901-5e27ad2546fc admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:10.188465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f4898-8ec4-42f4-8901-5e27ad2546fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:10.191001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f4898-8ec4-42f4-8901-5e27ad2546fc admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:10.232841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 884/1787] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:10 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:10.234080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff3bcb-9684-4504-96ea-2c2ff80c18c9 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:10.247474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff3bcb-9684-4504-96ea-2c2ff80c18c9 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:10.253387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b571d-dbea-4438-b22b-d09ed4f6077c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:10.254198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b571d-dbea-4438-b22b-d09ed4f6077c admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:10.254481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b571d-dbea-4438-b22b-d09ed4f6077c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:10.254751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b571d-dbea-4438-b22b-d09ed4f6077c admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:10.255447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b571d-dbea-4438-b22b-d09ed4f6077c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:10.257325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b571d-dbea-4438-b22b-d09ed4f6077c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:10.277776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 904/1788] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:46:10.279933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f4898-8ec4-42f4-8901-5e27ad2546fc admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:10.291448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f4898-8ec4-42f4-8901-5e27ad2546fc admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:10.296280 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d6060-4eaf-4c01-a9f7-f21f4d8dfe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:10.296942 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d6060-4eaf-4c01-a9f7-f21f4d8dfe78 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:10.297361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d6060-4eaf-4c01-a9f7-f21f4d8dfe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:10.297529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d6060-4eaf-4c01-a9f7-f21f4d8dfe78 admin admin] PATH_INFO: `/v3/projects/42b676b8248c407a8f9152a8333e8733/users/b87c753a13064ab4a09de769c7f67864/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:10.337021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d6060-4eaf-4c01-a9f7-f21f4d8dfe78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b676b8248c407a8f9152a8333e8733, user_id=b87c753a13064ab4a09de769c7f67864, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:10.340485 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d6060-4eaf-4c01-a9f7-f21f4d8dfe78 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:10.361759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 885/1789] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:10 2022] PUT /identity/v3/projects/42b676b8248c407a8f9152a8333e8733/users/b87c753a13064ab4a09de769c7f67864/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:10.365869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b571d-dbea-4438-b22b-d09ed4f6077c admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:10.379290 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b571d-dbea-4438-b22b-d09ed4f6077c admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:10.449673 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46dff23-ccd5-4fca-b1f4-75c13297c5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:10.450456 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dff23-ccd5-4fca-b1f4-75c13297c5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:10.451036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dff23-ccd5-4fca-b1f4-75c13297c5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:10.451528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dff23-ccd5-4fca-b1f4-75c13297c5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:10.452266 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46dff23-ccd5-4fca-b1f4-75c13297c5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:10.454053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46dff23-ccd5-4fca-b1f4-75c13297c5b1 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:10.464927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 905/1790] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:10.469325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d6060-4eaf-4c01-a9f7-f21f4d8dfe78 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:10.486550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d6060-4eaf-4c01-a9f7-f21f4d8dfe78 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:10.491433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7b629-f44d-41f5-a5e9-82b8f37234eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:10.492228 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7b629-f44d-41f5-a5e9-82b8f37234eb admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:10.493038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7b629-f44d-41f5-a5e9-82b8f37234eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:10.493310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7b629-f44d-41f5-a5e9-82b8f37234eb admin admin] PATH_INFO: `/v3/projects/42b676b8248c407a8f9152a8333e8733/users/b87c753a13064ab4a09de769c7f67864/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:10.495269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7b629-f44d-41f5-a5e9-82b8f37234eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b676b8248c407a8f9152a8333e8733, user_id=b87c753a13064ab4a09de769c7f67864, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:10.497956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7b629-f44d-41f5-a5e9-82b8f37234eb admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:10.510060 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 886/1791] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:10 2022] PUT /identity/v3/projects/42b676b8248c407a8f9152a8333e8733/users/b87c753a13064ab4a09de769c7f67864/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:10.521479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bae81-6cb0-47b5-be03-b598807242f9 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:10.521943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bae81-6cb0-47b5-be03-b598807242f9 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:10.522251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bae81-6cb0-47b5-be03-b598807242f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:10.546035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:10.556125 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-a65bae81-6cb0-47b5-be03-b598807242f9 None None] MFA Rules not processed for user `b87c753a13064ab4a09de769c7f67864`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:10.557900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a65bae81-6cb0-47b5-be03-b598807242f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:10.630419 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 906/1792] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:10.642057 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7b629-f44d-41f5-a5e9-82b8f37234eb admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:10.643736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c9e7b629-f44d-41f5-a5e9-82b8f37234eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:10.721463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7b629-f44d-41f5-a5e9-82b8f37234eb admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:10.725121 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e6824-fa5f-4e46-ad3d-6f409d760b9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:10.725762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e6824-fa5f-4e46-ad3d-6f409d760b9f service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:10.726041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e6824-fa5f-4e46-ad3d-6f409d760b9f service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:10.726308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e6824-fa5f-4e46-ad3d-6f409d760b9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:10.736800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e6824-fa5f-4e46-ad3d-6f409d760b9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:10.738312 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e6824-fa5f-4e46-ad3d-6f409d760b9f service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:10.757757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 887/1793] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:10 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:12.407159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bae81-6cb0-47b5-be03-b598807242f9 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:12.474681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bae81-6cb0-47b5-be03-b598807242f9 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:12.479780 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:12.480406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:12.480713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:12.480984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] PATH_INFO: `/v3/users/93c6829ab8da4c15a76b16af47637bb0` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:12.482307 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c6829ab8da4c15a76b16af47637bb0)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:12.484289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:12.536039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c6829ab8da4c15a76b16af47637bb0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:12.536710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c6829ab8da4c15a76b16af47637bb0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:12.544135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c6829ab8da4c15a76b16af47637bb0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:12.544781 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c6829ab8da4c15a76b16af47637bb0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:12.556397 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c6829ab8da4c15a76b16af47637bb0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:12.570838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 907/1794] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:12 2022] DELETE /identity/v3/users/93c6829ab8da4c15a76b16af47637bb0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:12.575423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e96081-398a-445d-ad8b-383bac4058bd None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:12.575936 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e96081-398a-445d-ad8b-383bac4058bd None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:12.576032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e96081-398a-445d-ad8b-383bac4058bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:12.597895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-68e96081-398a-445d-ad8b-383bac4058bd None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:12.599117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-68e96081-398a-445d-ad8b-383bac4058bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:12.657252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 888/1795] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:12.667457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:12.668870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:12.687226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:12.728412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15dafa-3fdf-4f75-8958-365f6ef7085d admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:12.732006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ca4ad-6e54-4886-b1cd-2f9948008ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:12.732605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ca4ad-6e54-4886-b1cd-2f9948008ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:12.732884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ca4ad-6e54-4886-b1cd-2f9948008ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:12.733156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ca4ad-6e54-4886-b1cd-2f9948008ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:12.742228 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ca4ad-6e54-4886-b1cd-2f9948008ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:12.743663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ca4ad-6e54-4886-b1cd-2f9948008ba1 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:12.752978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 908/1796] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:13.019813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e96081-398a-445d-ad8b-383bac4058bd None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:13.021250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-68e96081-398a-445d-ad8b-383bac4058bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:13.033861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e96081-398a-445d-ad8b-383bac4058bd None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:13.038616 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:13.039755 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:13.040040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:13.040310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] PATH_INFO: `/v3/projects/6662669c78994181bfe4b5dd7f9b1ec9` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:13.041436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6662669c78994181bfe4b5dd7f9b1ec9)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:13.043494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:13.085340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:13.094601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6662669c78994181bfe4b5dd7f9b1ec9', 'request_id': 'req-0bd01305-0046-49c8-bef0-0acf8dc2eb73'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.103629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6662669c78994181bfe4b5dd7f9b1ec9', 'request_id': 'req-0bd01305-0046-49c8-bef0-0acf8dc2eb73'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.109431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6662669c78994181bfe4b5dd7f9b1ec9', 'request_id': 'req-0bd01305-0046-49c8-bef0-0acf8dc2eb73'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.122605 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] The token cache is being invalidate because project 6662669c78994181bfe4b5dd7f9b1ec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:13.122896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.123520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.125008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 889/1797] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:13 2022] DELETE /identity/v3/projects/6662669c78994181bfe4b5dd7f9b1ec9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:13.129004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ca4ad-6e54-4886-b1cd-2f9948008ba1 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:13.138452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ca4ad-6e54-4886-b1cd-2f9948008ba1 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:13.201361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b4804-b91e-4138-b3fd-3b4a8b57d319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:13.201361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b4804-b91e-4138-b3fd-3b4a8b57d319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:13.201361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b4804-b91e-4138-b3fd-3b4a8b57d319 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:13.201361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b4804-b91e-4138-b3fd-3b4a8b57d319 admin admin] PATH_INFO: `/v3/users/b87c753a13064ab4a09de769c7f67864` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:13.202804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b4804-b91e-4138-b3fd-3b4a8b57d319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b87c753a13064ab4a09de769c7f67864)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:13.204785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b4804-b91e-4138-b3fd-3b4a8b57d319 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:13.252854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f72b4804-b91e-4138-b3fd-3b4a8b57d319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b87c753a13064ab4a09de769c7f67864'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.253543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f72b4804-b91e-4138-b3fd-3b4a8b57d319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b87c753a13064ab4a09de769c7f67864'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.261639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f72b4804-b91e-4138-b3fd-3b4a8b57d319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b87c753a13064ab4a09de769c7f67864'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.262327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f72b4804-b91e-4138-b3fd-3b4a8b57d319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b87c753a13064ab4a09de769c7f67864'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.275365 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f72b4804-b91e-4138-b3fd-3b4a8b57d319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b87c753a13064ab4a09de769c7f67864'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.292729 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 909/1798] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:13 2022] DELETE /identity/v3/users/b87c753a13064ab4a09de769c7f67864 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:13.745255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:13.747038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:13.761266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd01305-0046-49c8-bef0-0acf8dc2eb73 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:13.820249 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:13.821408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:13.821713 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:13.821999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8 admin admin] PATH_INFO: `/v3/projects/42b676b8248c407a8f9152a8333e8733` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:13.823125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b676b8248c407a8f9152a8333e8733)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:13.825043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:13.874205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b676b8248c407a8f9152a8333e8733', 'request_id': 'req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.880335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b676b8248c407a8f9152a8333e8733', 'request_id': 'req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.886994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b676b8248c407a8f9152a8333e8733', 'request_id': 'req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.899821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8 admin admin] The token cache is being invalidate because project 42b676b8248c407a8f9152a8333e8733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:13.900108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.900728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-37ee89b0-6872-41c8-9ced-17d5fa7db1f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:13.902285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 890/1799] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:13 2022] DELETE /identity/v3/projects/42b676b8248c407a8f9152a8333e8733 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:13.907101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98d766-5196-4739-afc6-482ab582cbc1 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:13.907579 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98d766-5196-4739-afc6-482ab582cbc1 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:13.907694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98d766-5196-4739-afc6-482ab582cbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:13.926730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-bf98d766-5196-4739-afc6-482ab582cbc1 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:13.927939 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-bf98d766-5196-4739-afc6-482ab582cbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:13.959827 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:13.988146 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 910/1800] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:13.995756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536250b-3720-4942-8bbb-9dbca89476cf None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:13.996040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536250b-3720-4942-8bbb-9dbca89476cf None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:13.996311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536250b-3720-4942-8bbb-9dbca89476cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:14.017286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c536250b-3720-4942-8bbb-9dbca89476cf None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:14.018552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c536250b-3720-4942-8bbb-9dbca89476cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:14.026985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 891/1801] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:14.031063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98d766-5196-4739-afc6-482ab582cbc1 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:14.040673 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98d766-5196-4739-afc6-482ab582cbc1 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:14.045274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cbe1b-2d4e-4963-b32a-1769c9edc526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:14.045893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cbe1b-2d4e-4963-b32a-1769c9edc526 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:14.046175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cbe1b-2d4e-4963-b32a-1769c9edc526 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:14.046473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cbe1b-2d4e-4963-b32a-1769c9edc526 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:14.047110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cbe1b-2d4e-4963-b32a-1769c9edc526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:14.047414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cbe1b-2d4e-4963-b32a-1769c9edc526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:14.049011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3cbe1b-2d4e-4963-b32a-1769c9edc526 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:14.061094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 911/1802] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:46:14.065679 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536250b-3720-4942-8bbb-9dbca89476cf None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:14.075949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536250b-3720-4942-8bbb-9dbca89476cf None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:14.080534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf8b8c-3c92-49ea-bfbb-da62c3952022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:14.081164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf8b8c-3c92-49ea-bfbb-da62c3952022 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:14.081445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf8b8c-3c92-49ea-bfbb-da62c3952022 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:14.081752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf8b8c-3c92-49ea-bfbb-da62c3952022 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:14.082387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf8b8c-3c92-49ea-bfbb-da62c3952022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:14.084289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf8b8c-3c92-49ea-bfbb-da62c3952022 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:14.115808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 892/1803] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:14 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:46:14.117089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cbe1b-2d4e-4963-b32a-1769c9edc526 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:14.132731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cbe1b-2d4e-4963-b32a-1769c9edc526 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:14.206820 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc171e-2fb5-4f59-8bf7-e77ee13e4dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:14.207429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc171e-2fb5-4f59-8bf7-e77ee13e4dfd admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:14.207704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc171e-2fb5-4f59-8bf7-e77ee13e4dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:14.207976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc171e-2fb5-4f59-8bf7-e77ee13e4dfd admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:14.208599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc171e-2fb5-4f59-8bf7-e77ee13e4dfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:14.210567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc171e-2fb5-4f59-8bf7-e77ee13e4dfd admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:14.233942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 912/1804] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:14 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:46:14.239355 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf8b8c-3c92-49ea-bfbb-da62c3952022 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:14.256846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf8b8c-3c92-49ea-bfbb-da62c3952022 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:14.265998 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0a3e2-4f34-4e18-9333-6b6e463ccfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:14.267075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0a3e2-4f34-4e18-9333-6b6e463ccfc7 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:14.267788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0a3e2-4f34-4e18-9333-6b6e463ccfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:14.268314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0a3e2-4f34-4e18-9333-6b6e463ccfc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:14.269183 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0a3e2-4f34-4e18-9333-6b6e463ccfc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:14.271565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0a3e2-4f34-4e18-9333-6b6e463ccfc7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:14.292591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 893/1805] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:46:14.295307 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc171e-2fb5-4f59-8bf7-e77ee13e4dfd admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:14.306192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc171e-2fb5-4f59-8bf7-e77ee13e4dfd admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:14.311860 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bb78e-5839-4c29-a907-5f63fda5ed3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:14.312479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bb78e-5839-4c29-a907-5f63fda5ed3d admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:14.312756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bb78e-5839-4c29-a907-5f63fda5ed3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:14.313028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bb78e-5839-4c29-a907-5f63fda5ed3d admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:14.313670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40bb78e-5839-4c29-a907-5f63fda5ed3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:14.315301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40bb78e-5839-4c29-a907-5f63fda5ed3d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:14.329743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 913/1806] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:14.334220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0a3e2-4f34-4e18-9333-6b6e463ccfc7 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:14.344273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0a3e2-4f34-4e18-9333-6b6e463ccfc7 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:14.349189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24e1cc-094c-456c-9167-fe5f06fc67a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:14.349902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24e1cc-094c-456c-9167-fe5f06fc67a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:14.350189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24e1cc-094c-456c-9167-fe5f06fc67a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:14.350464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c24e1cc-094c-456c-9167-fe5f06fc67a7 admin admin] PATH_INFO: `/v3/projects/965735db7bf74f14a696b3cb2b5598e4/users/bd483a9ec8d645d4a08d1581eaf9f746/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:14.365454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24e1cc-094c-456c-9167-fe5f06fc67a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=965735db7bf74f14a696b3cb2b5598e4, user_id=bd483a9ec8d645d4a08d1581eaf9f746, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:14.369101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c24e1cc-094c-456c-9167-fe5f06fc67a7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:14.371289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:14.391388 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 894/1807] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:14 2022] PUT /identity/v3/projects/965735db7bf74f14a696b3cb2b5598e4/users/bd483a9ec8d645d4a08d1581eaf9f746/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:14.401743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d998a4-fc34-4a94-aa47-ea605ff73f62 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:14.402041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d998a4-fc34-4a94-aa47-ea605ff73f62 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:14.402316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d998a4-fc34-4a94-aa47-ea605ff73f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:14.433034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b7d998a4-fc34-4a94-aa47-ea605ff73f62 None None] MFA Rules not processed for user `bd483a9ec8d645d4a08d1581eaf9f746`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:14.434349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b7d998a4-fc34-4a94-aa47-ea605ff73f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:14.520907 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 914/1808] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:14.521808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24e1cc-094c-456c-9167-fe5f06fc67a7 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:14.597630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c24e1cc-094c-456c-9167-fe5f06fc67a7 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:14.603486 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc1d40-57e2-49c1-aea3-b7b54370651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:14.603702 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc1d40-57e2-49c1-aea3-b7b54370651f admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:14.603993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc1d40-57e2-49c1-aea3-b7b54370651f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:14.604269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc1d40-57e2-49c1-aea3-b7b54370651f admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:14.605103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc1d40-57e2-49c1-aea3-b7b54370651f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:14.606842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc1d40-57e2-49c1-aea3-b7b54370651f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:14.642579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 895/1809] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:14 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:46:14.644650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d998a4-fc34-4a94-aa47-ea605ff73f62 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:14.674334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d998a4-fc34-4a94-aa47-ea605ff73f62 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:14.718051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:14.771777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb1b56-5b5e-4537-b465-da66d2554f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:14.772570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb1b56-5b5e-4537-b465-da66d2554f19 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:14.773017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb1b56-5b5e-4537-b465-da66d2554f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:14.774392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb1b56-5b5e-4537-b465-da66d2554f19 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:14.774835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb1b56-5b5e-4537-b465-da66d2554f19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:14.790106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb1b56-5b5e-4537-b465-da66d2554f19 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:14.824661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 915/1810] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:14 2022] POST /identity/v3/users => generated 300 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:14.828646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc1d40-57e2-49c1-aea3-b7b54370651f admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:14.840739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc1d40-57e2-49c1-aea3-b7b54370651f admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:14.844306 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2eb679-ddb5-47d9-b874-1f814520e7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:14.845031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eb679-ddb5-47d9-b874-1f814520e7fe admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:14.845453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eb679-ddb5-47d9-b874-1f814520e7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:14.845755 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eb679-ddb5-47d9-b874-1f814520e7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:14.846405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2eb679-ddb5-47d9-b874-1f814520e7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:14.848223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2eb679-ddb5-47d9-b874-1f814520e7fe admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:14.863980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 896/1811] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:46:14.868004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb1b56-5b5e-4537-b465-da66d2554f19 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:14.885384 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb1b56-5b5e-4537-b465-da66d2554f19 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:14.890876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5e337-d5cc-4734-892a-f7258f6f5de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:14.891565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e337-d5cc-4734-892a-f7258f6f5de1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:14.891967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e337-d5cc-4734-892a-f7258f6f5de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:14.892503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5e337-d5cc-4734-892a-f7258f6f5de1 admin admin] PATH_INFO: `/v3/projects/61a33abe30f34c11abf94393ef76384c/users/1c15c9feb396407283c5e88794f523d4/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:14.909434 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5e337-d5cc-4734-892a-f7258f6f5de1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a33abe30f34c11abf94393ef76384c, user_id=1c15c9feb396407283c5e88794f523d4, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:14.912681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5e337-d5cc-4734-892a-f7258f6f5de1 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:14.933710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 916/1812] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:14 2022] PUT /identity/v3/projects/61a33abe30f34c11abf94393ef76384c/users/1c15c9feb396407283c5e88794f523d4/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:14.934956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2eb679-ddb5-47d9-b874-1f814520e7fe admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:14.947083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2eb679-ddb5-47d9-b874-1f814520e7fe admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:15.075193 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c7703-13e6-45a1-b091-7179ba02664d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:15.075825 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c7703-13e6-45a1-b091-7179ba02664d admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:15.076106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c7703-13e6-45a1-b091-7179ba02664d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:15.076384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c7703-13e6-45a1-b091-7179ba02664d admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:15.077038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c7703-13e6-45a1-b091-7179ba02664d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:15.078677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c7703-13e6-45a1-b091-7179ba02664d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:15.104447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 897/1813] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:15.108421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5e337-d5cc-4734-892a-f7258f6f5de1 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:15.120062 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5e337-d5cc-4734-892a-f7258f6f5de1 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:15.125181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d0428-3307-42cf-865e-8c45a84b98f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:15.126136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0428-3307-42cf-865e-8c45a84b98f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:15.126509 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0428-3307-42cf-865e-8c45a84b98f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:15.126961 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0428-3307-42cf-865e-8c45a84b98f3 admin admin] PATH_INFO: `/v3/projects/61a33abe30f34c11abf94393ef76384c/users/1c15c9feb396407283c5e88794f523d4/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:15.128986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d0428-3307-42cf-865e-8c45a84b98f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a33abe30f34c11abf94393ef76384c, user_id=1c15c9feb396407283c5e88794f523d4, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:15.131917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d0428-3307-42cf-865e-8c45a84b98f3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:15.182334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 917/1814] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:15 2022] PUT /identity/v3/projects/61a33abe30f34c11abf94393ef76384c/users/1c15c9feb396407283c5e88794f523d4/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:15.185183 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab93ed9-e76c-4b58-a925-bdb3068caa8e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:15.185530 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab93ed9-e76c-4b58-a925-bdb3068caa8e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:15.185814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab93ed9-e76c-4b58-a925-bdb3068caa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:15.228077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-cab93ed9-e76c-4b58-a925-bdb3068caa8e None None] MFA Rules not processed for user `1c15c9feb396407283c5e88794f523d4`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:15.229500 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-cab93ed9-e76c-4b58-a925-bdb3068caa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:15.284663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:15.311742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 898/1815] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:15 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:15.323523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d0428-3307-42cf-865e-8c45a84b98f3 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:15.325600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-dc7d0428-3307-42cf-865e-8c45a84b98f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:15.391033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d0428-3307-42cf-865e-8c45a84b98f3 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:15.395233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e8841-71e2-45a0-be8a-0f727d32cf2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:15.395949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e8841-71e2-45a0-be8a-0f727d32cf2a service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:15.396422 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e8841-71e2-45a0-be8a-0f727d32cf2a service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:15.396743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e8841-71e2-45a0-be8a-0f727d32cf2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:15.407267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e8841-71e2-45a0-be8a-0f727d32cf2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:15.408725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e8841-71e2-45a0-be8a-0f727d32cf2a service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:15.420341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 918/1816] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:15 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:15.450132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab93ed9-e76c-4b58-a925-bdb3068caa8e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:15.512949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab93ed9-e76c-4b58-a925-bdb3068caa8e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:15.517652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:15.518306 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:15.518587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:15.518862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] PATH_INFO: `/v3/users/bd483a9ec8d645d4a08d1581eaf9f746` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:15.519994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd483a9ec8d645d4a08d1581eaf9f746)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:15.521955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:15.570010 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd483a9ec8d645d4a08d1581eaf9f746'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:15.571265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd483a9ec8d645d4a08d1581eaf9f746'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:15.582629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd483a9ec8d645d4a08d1581eaf9f746'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:15.589473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd483a9ec8d645d4a08d1581eaf9f746'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:15.590120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd483a9ec8d645d4a08d1581eaf9f746'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:15.603549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 899/1817] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:15 2022] DELETE /identity/v3/users/bd483a9ec8d645d4a08d1581eaf9f746 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:15.608118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643780f-f871-4f46-856a-6964b553143b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:15.608408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643780f-f871-4f46-856a-6964b553143b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:15.608690 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643780f-f871-4f46-856a-6964b553143b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:15.627539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-3643780f-f871-4f46-856a-6964b553143b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:15.628736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3643780f-f871-4f46-856a-6964b553143b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:15.686780 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 919/1818] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:15.697027 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:15.698579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:15.753745 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c64ade-2745-4e2a-a3a7-7d47a8f47b60 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:15.757255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd23ec2-d9b7-4d1d-b56a-2d70dde68d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:15.758403 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd23ec2-d9b7-4d1d-b56a-2d70dde68d4f service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:15.758695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd23ec2-d9b7-4d1d-b56a-2d70dde68d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:15.758968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd23ec2-d9b7-4d1d-b56a-2d70dde68d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:15.768522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd23ec2-d9b7-4d1d-b56a-2d70dde68d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:15.770085 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd23ec2-d9b7-4d1d-b56a-2d70dde68d4f service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:15.779483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 900/1819] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:16.059302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643780f-f871-4f46-856a-6964b553143b None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:16.071170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3643780f-f871-4f46-856a-6964b553143b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:16.094049 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643780f-f871-4f46-856a-6964b553143b None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:16.099636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:16.100283 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:16.100568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:16.100842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] PATH_INFO: `/v3/projects/965735db7bf74f14a696b3cb2b5598e4` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:16.102634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] RBAC: Authorizing `identity:delete_project(project_id=965735db7bf74f14a696b3cb2b5598e4)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:16.105538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:16.138040 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:16.182309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '965735db7bf74f14a696b3cb2b5598e4', 'request_id': 'req-1bfb1795-9a8b-450b-afc2-612f3f54cadb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.191453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '965735db7bf74f14a696b3cb2b5598e4', 'request_id': 'req-1bfb1795-9a8b-450b-afc2-612f3f54cadb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.199841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '965735db7bf74f14a696b3cb2b5598e4', 'request_id': 'req-1bfb1795-9a8b-450b-afc2-612f3f54cadb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.213363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] The token cache is being invalidate because project 965735db7bf74f14a696b3cb2b5598e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:16.213659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.214356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.216368 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 920/1820] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:16 2022] DELETE /identity/v3/projects/965735db7bf74f14a696b3cb2b5598e4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:16.220631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd23ec2-d9b7-4d1d-b56a-2d70dde68d4f service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:16.233334 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd23ec2-d9b7-4d1d-b56a-2d70dde68d4f service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:16.241723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:16.323285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac94228-9b29-420a-864d-2a4344b8c826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:16.323940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac94228-9b29-420a-864d-2a4344b8c826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:16.324223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac94228-9b29-420a-864d-2a4344b8c826 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:16.324505 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac94228-9b29-420a-864d-2a4344b8c826 admin admin] PATH_INFO: `/v3/users/1c15c9feb396407283c5e88794f523d4` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:16.325738 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac94228-9b29-420a-864d-2a4344b8c826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c15c9feb396407283c5e88794f523d4)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:16.327725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac94228-9b29-420a-864d-2a4344b8c826 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:16.384319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fac94228-9b29-420a-864d-2a4344b8c826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c15c9feb396407283c5e88794f523d4'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.385159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fac94228-9b29-420a-864d-2a4344b8c826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c15c9feb396407283c5e88794f523d4'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.398825 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fac94228-9b29-420a-864d-2a4344b8c826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c15c9feb396407283c5e88794f523d4'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.406602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fac94228-9b29-420a-864d-2a4344b8c826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c15c9feb396407283c5e88794f523d4'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.407405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fac94228-9b29-420a-864d-2a4344b8c826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c15c9feb396407283c5e88794f523d4'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.420908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 901/1821] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:16 2022] DELETE /identity/v3/users/1c15c9feb396407283c5e88794f523d4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:16.694182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:16.696539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:16.712827 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb1795-9a8b-450b-afc2-612f3f54cadb admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:16.772604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69060fa3-2761-41be-a7ca-85f012304018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:16.773239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69060fa3-2761-41be-a7ca-85f012304018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:16.774124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69060fa3-2761-41be-a7ca-85f012304018 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:16.774404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69060fa3-2761-41be-a7ca-85f012304018 admin admin] PATH_INFO: `/v3/projects/61a33abe30f34c11abf94393ef76384c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:16.775521 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69060fa3-2761-41be-a7ca-85f012304018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a33abe30f34c11abf94393ef76384c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:16.777431 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69060fa3-2761-41be-a7ca-85f012304018 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:16.822176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-69060fa3-2761-41be-a7ca-85f012304018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a33abe30f34c11abf94393ef76384c', 'request_id': 'req-69060fa3-2761-41be-a7ca-85f012304018'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.828249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-69060fa3-2761-41be-a7ca-85f012304018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a33abe30f34c11abf94393ef76384c', 'request_id': 'req-69060fa3-2761-41be-a7ca-85f012304018'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.833790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-69060fa3-2761-41be-a7ca-85f012304018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a33abe30f34c11abf94393ef76384c', 'request_id': 'req-69060fa3-2761-41be-a7ca-85f012304018'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.847492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-69060fa3-2761-41be-a7ca-85f012304018 admin admin] The token cache is being invalidate because project 61a33abe30f34c11abf94393ef76384c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:16.847787 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-69060fa3-2761-41be-a7ca-85f012304018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.848425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-69060fa3-2761-41be-a7ca-85f012304018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:16.849958 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 921/1822] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:16 2022] DELETE /identity/v3/projects/61a33abe30f34c11abf94393ef76384c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:16.854974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0188c-3ad8-4add-8f7e-bce95d6a6fbf None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:16.855258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0188c-3ad8-4add-8f7e-bce95d6a6fbf None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:16.855533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0188c-3ad8-4add-8f7e-bce95d6a6fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:16.876065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-4ac0188c-3ad8-4add-8f7e-bce95d6a6fbf None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:16.877436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4ac0188c-3ad8-4add-8f7e-bce95d6a6fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:16.939416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 902/1823] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:16.947583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f31a43-a388-4a34-b728-167bee40fb93 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:16.947905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f31a43-a388-4a34-b728-167bee40fb93 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:16.948186 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f31a43-a388-4a34-b728-167bee40fb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:16.968045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-f0f31a43-a388-4a34-b728-167bee40fb93 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:16.969247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f0f31a43-a388-4a34-b728-167bee40fb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:16.977395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 922/1824] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:16.981404 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0188c-3ad8-4add-8f7e-bce95d6a6fbf None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:16.990891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0188c-3ad8-4add-8f7e-bce95d6a6fbf None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:16.995437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fadb2-1c09-4116-b2d0-0a1d40d4c449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:16.996054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fadb2-1c09-4116-b2d0-0a1d40d4c449 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:16.996331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fadb2-1c09-4116-b2d0-0a1d40d4c449 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:16.996599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fadb2-1c09-4116-b2d0-0a1d40d4c449 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:16.997236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fadb2-1c09-4116-b2d0-0a1d40d4c449 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:16.997542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fadb2-1c09-4116-b2d0-0a1d40d4c449 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:16.999159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fadb2-1c09-4116-b2d0-0a1d40d4c449 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.011268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 903/1825] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:46:17.015486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f31a43-a388-4a34-b728-167bee40fb93 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:17.025141 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f31a43-a388-4a34-b728-167bee40fb93 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:17.029604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002daee8-bb0c-4507-ad82-a2397e0e852b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:17.030210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002daee8-bb0c-4507-ad82-a2397e0e852b admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.030485 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002daee8-bb0c-4507-ad82-a2397e0e852b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.030755 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002daee8-bb0c-4507-ad82-a2397e0e852b admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.031376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002daee8-bb0c-4507-ad82-a2397e0e852b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:17.033256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002daee8-bb0c-4507-ad82-a2397e0e852b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.059776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 923/1826] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:17 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:17.063877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fadb2-1c09-4116-b2d0-0a1d40d4c449 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:17.074379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fadb2-1c09-4116-b2d0-0a1d40d4c449 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:17.086205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:17.142847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3c928-1ac7-4354-807a-7af89f131674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:17.143465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3c928-1ac7-4354-807a-7af89f131674 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.143740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3c928-1ac7-4354-807a-7af89f131674 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.144006 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3c928-1ac7-4354-807a-7af89f131674 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.144646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3c928-1ac7-4354-807a-7af89f131674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:17.146606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3c928-1ac7-4354-807a-7af89f131674 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.167073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 904/1827] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:17 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:17.171392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002daee8-bb0c-4507-ad82-a2397e0e852b admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:17.182138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002daee8-bb0c-4507-ad82-a2397e0e852b admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:17.188341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2565d4-a5f5-42d2-a287-56444aa8df33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:17.188955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2565d4-a5f5-42d2-a287-56444aa8df33 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.189233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2565d4-a5f5-42d2-a287-56444aa8df33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.189503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2565d4-a5f5-42d2-a287-56444aa8df33 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.190216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2565d4-a5f5-42d2-a287-56444aa8df33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:17.192251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2565d4-a5f5-42d2-a287-56444aa8df33 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.202776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 924/1828] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:17.206791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3c928-1ac7-4354-807a-7af89f131674 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:17.216664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3c928-1ac7-4354-807a-7af89f131674 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:17.221473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15d79e-f35d-42b9-9f77-f41db2bc5518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:17.222183 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15d79e-f35d-42b9-9f77-f41db2bc5518 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.222390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15d79e-f35d-42b9-9f77-f41db2bc5518 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.222662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15d79e-f35d-42b9-9f77-f41db2bc5518 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.223296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15d79e-f35d-42b9-9f77-f41db2bc5518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:17.224870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15d79e-f35d-42b9-9f77-f41db2bc5518 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.235442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 905/1829] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:46:17.239721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2565d4-a5f5-42d2-a287-56444aa8df33 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:17.245553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:17.250750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2565d4-a5f5-42d2-a287-56444aa8df33 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:17.256748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a81c2e-8ef7-4933-bb20-e1198cc73eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:17.257551 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a81c2e-8ef7-4933-bb20-e1198cc73eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.258605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a81c2e-8ef7-4933-bb20-e1198cc73eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.258887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a81c2e-8ef7-4933-bb20-e1198cc73eb3 admin admin] PATH_INFO: `/v3/projects/cb03cd19940a4860b827dc30d3db8fc8/users/6a5e420ac6c24bb0a782f6f3820f60a9/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.274162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a81c2e-8ef7-4933-bb20-e1198cc73eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb03cd19940a4860b827dc30d3db8fc8, user_id=6a5e420ac6c24bb0a782f6f3820f60a9, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:17.276875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a81c2e-8ef7-4933-bb20-e1198cc73eb3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.292473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 925/1830] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:17 2022] PUT /identity/v3/projects/cb03cd19940a4860b827dc30d3db8fc8/users/6a5e420ac6c24bb0a782f6f3820f60a9/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:17.305263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778aaaf-550c-46cc-a1de-bcd18c6e0872 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.305618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778aaaf-550c-46cc-a1de-bcd18c6e0872 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.307227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778aaaf-550c-46cc-a1de-bcd18c6e0872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.336212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-b778aaaf-550c-46cc-a1de-bcd18c6e0872 None None] MFA Rules not processed for user `6a5e420ac6c24bb0a782f6f3820f60a9`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:17.337847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b778aaaf-550c-46cc-a1de-bcd18c6e0872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:17.424609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 906/1831] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:17.429297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a81c2e-8ef7-4933-bb20-e1198cc73eb3 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:17.480904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2779772-a4ee-4d73-92a1-d5202ba955b5 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.481055 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2779772-a4ee-4d73-92a1-d5202ba955b5 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.481675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2779772-a4ee-4d73-92a1-d5202ba955b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.504624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a2779772-a4ee-4d73-92a1-d5202ba955b5 None None] MFA Rules not processed for user `50d3b75cbc5c4d1686f4d93248e04746`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:17.505920 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a2779772-a4ee-4d73-92a1-d5202ba955b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:17.537254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a81c2e-8ef7-4933-bb20-e1198cc73eb3 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:17.543643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830c643-24a5-4f63-bd90-5da79ce7d63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:17.544262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830c643-24a5-4f63-bd90-5da79ce7d63d admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.545047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830c643-24a5-4f63-bd90-5da79ce7d63d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.545334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830c643-24a5-4f63-bd90-5da79ce7d63d admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.546020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830c643-24a5-4f63-bd90-5da79ce7d63d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:17.548208 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830c643-24a5-4f63-bd90-5da79ce7d63d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.576997 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 907/1832] 10.222.0.46 () {58 vars in 1015 bytes} [Tue Oct 4 08:46:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:17.582891 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 926/1833] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:17 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:17.586909 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2779772-a4ee-4d73-92a1-d5202ba955b5 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:17.599036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2779772-a4ee-4d73-92a1-d5202ba955b5 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:17.609128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830c643-24a5-4f63-bd90-5da79ce7d63d admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:17.614433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e830c643-24a5-4f63-bd90-5da79ce7d63d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:17.668368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:17.694187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830c643-24a5-4f63-bd90-5da79ce7d63d admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:17.696355 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99130df4-8824-44be-a873-210784e13f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:17.696985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99130df4-8824-44be-a873-210784e13f81 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.697268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99130df4-8824-44be-a873-210784e13f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.697539 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99130df4-8824-44be-a873-210784e13f81 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.698281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99130df4-8824-44be-a873-210784e13f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:17.698382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3221935-ce6b-4eba-9a89-030212a8c9a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:17.699020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3221935-ce6b-4eba-9a89-030212a8c9a2 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.699281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3221935-ce6b-4eba-9a89-030212a8c9a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.699573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3221935-ce6b-4eba-9a89-030212a8c9a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.700146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99130df4-8824-44be-a873-210784e13f81 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.709071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3221935-ce6b-4eba-9a89-030212a8c9a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:17.710640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3221935-ce6b-4eba-9a89-030212a8c9a2 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.719499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 927/1834] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:17.725733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 908/1835] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:17 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:17.728554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3221935-ce6b-4eba-9a89-030212a8c9a2 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:17.739269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3221935-ce6b-4eba-9a89-030212a8c9a2 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:17.744358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5b47c-9f4c-4e38-b456-8416495d1903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:17.744958 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5b47c-9f4c-4e38-b456-8416495d1903 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.745234 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5b47c-9f4c-4e38-b456-8416495d1903 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.745506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5b47c-9f4c-4e38-b456-8416495d1903 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.746240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5b47c-9f4c-4e38-b456-8416495d1903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:17.747909 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5b47c-9f4c-4e38-b456-8416495d1903 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.758833 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 928/1836] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:46:17.763041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99130df4-8824-44be-a873-210784e13f81 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:17.774086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99130df4-8824-44be-a873-210784e13f81 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:17.778905 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5834ef-f90a-4d87-ab3a-4fc1cfa8a9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:17.779568 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5834ef-f90a-4d87-ab3a-4fc1cfa8a9b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.779847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5834ef-f90a-4d87-ab3a-4fc1cfa8a9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.780141 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5834ef-f90a-4d87-ab3a-4fc1cfa8a9b2 admin admin] PATH_INFO: `/v3/projects/8ab473d955a34e2494f6897d2948e61a/users/3b23aed057db43feb9ca8f6db985f1b9/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.795654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5834ef-f90a-4d87-ab3a-4fc1cfa8a9b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab473d955a34e2494f6897d2948e61a, user_id=3b23aed057db43feb9ca8f6db985f1b9, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:17.798610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5834ef-f90a-4d87-ab3a-4fc1cfa8a9b2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.813893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 909/1837] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:17 2022] PUT /identity/v3/projects/8ab473d955a34e2494f6897d2948e61a/users/3b23aed057db43feb9ca8f6db985f1b9/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:17.817771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5b47c-9f4c-4e38-b456-8416495d1903 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:17.828453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5b47c-9f4c-4e38-b456-8416495d1903 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:17.892494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:17.901803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a70f2a-e9b2-4c70-985a-a57682862faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:17.902478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a70f2a-e9b2-4c70-985a-a57682862faa admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.902767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a70f2a-e9b2-4c70-985a-a57682862faa admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.903128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a70f2a-e9b2-4c70-985a-a57682862faa admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.903797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a70f2a-e9b2-4c70-985a-a57682862faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:17.906523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a70f2a-e9b2-4c70-985a-a57682862faa admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.944265 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 929/1838] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:17.948899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5834ef-f90a-4d87-ab3a-4fc1cfa8a9b2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:17.967061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5834ef-f90a-4d87-ab3a-4fc1cfa8a9b2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:17.979588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035d51c-b3e8-4675-bfae-dc056a805de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:17.980973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035d51c-b3e8-4675-bfae-dc056a805de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:17.980973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035d51c-b3e8-4675-bfae-dc056a805de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:17.981168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035d51c-b3e8-4675-bfae-dc056a805de7 admin admin] PATH_INFO: `/v3/projects/8ab473d955a34e2494f6897d2948e61a/users/3b23aed057db43feb9ca8f6db985f1b9/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:17.983268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035d51c-b3e8-4675-bfae-dc056a805de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab473d955a34e2494f6897d2948e61a, user_id=3b23aed057db43feb9ca8f6db985f1b9, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:17.986104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035d51c-b3e8-4675-bfae-dc056a805de7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:17.998753 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 910/1839] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:17 2022] PUT /identity/v3/projects/8ab473d955a34e2494f6897d2948e61a/users/3b23aed057db43feb9ca8f6db985f1b9/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:18.008758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e57736-4bd9-4f84-af4f-03a03cf4aa4c None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:18.009162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e57736-4bd9-4f84-af4f-03a03cf4aa4c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:18.010516 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e57736-4bd9-4f84-af4f-03a03cf4aa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:18.040968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-c9e57736-4bd9-4f84-af4f-03a03cf4aa4c None None] MFA Rules not processed for user `3b23aed057db43feb9ca8f6db985f1b9`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:18.042275 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c9e57736-4bd9-4f84-af4f-03a03cf4aa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:18.126987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 930/1840] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:18.140302 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035d51c-b3e8-4675-bfae-dc056a805de7 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:18.142757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e035d51c-b3e8-4675-bfae-dc056a805de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:18.213711 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035d51c-b3e8-4675-bfae-dc056a805de7 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:18.217463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784be34-7748-4793-abd3-1d8c5fa4e3fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:18.218116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784be34-7748-4793-abd3-1d8c5fa4e3fe service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:18.218424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784be34-7748-4793-abd3-1d8c5fa4e3fe service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:18.218702 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784be34-7748-4793-abd3-1d8c5fa4e3fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:18.230872 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784be34-7748-4793-abd3-1d8c5fa4e3fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:18.232335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784be34-7748-4793-abd3-1d8c5fa4e3fe service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:18.242670 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 911/1841] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:18.348966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e57736-4bd9-4f84-af4f-03a03cf4aa4c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:18.446289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e57736-4bd9-4f84-af4f-03a03cf4aa4c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:18.452840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a5c55-fd2f-4471-8bd6-1cbda3a8bb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:18.453531 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a5c55-fd2f-4471-8bd6-1cbda3a8bb8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:18.453840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a5c55-fd2f-4471-8bd6-1cbda3a8bb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:18.454239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a5c55-fd2f-4471-8bd6-1cbda3a8bb8e admin admin] PATH_INFO: `/v3/users/6a5e420ac6c24bb0a782f6f3820f60a9` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:18.456236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a5c55-fd2f-4471-8bd6-1cbda3a8bb8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a5e420ac6c24bb0a782f6f3820f60a9)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:18.458480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a5c55-fd2f-4471-8bd6-1cbda3a8bb8e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:18.550178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2a6a5c55-fd2f-4471-8bd6-1cbda3a8bb8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a5e420ac6c24bb0a782f6f3820f60a9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:18.565285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2a6a5c55-fd2f-4471-8bd6-1cbda3a8bb8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a5e420ac6c24bb0a782f6f3820f60a9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:18.583106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2a6a5c55-fd2f-4471-8bd6-1cbda3a8bb8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a5e420ac6c24bb0a782f6f3820f60a9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:18.584819 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2a6a5c55-fd2f-4471-8bd6-1cbda3a8bb8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a5e420ac6c24bb0a782f6f3820f60a9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:18.620007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2a6a5c55-fd2f-4471-8bd6-1cbda3a8bb8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a5e420ac6c24bb0a782f6f3820f60a9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:18.624266 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:18.644966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 931/1842] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:18 2022] DELETE /identity/v3/users/6a5e420ac6c24bb0a782f6f3820f60a9 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:18.654195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322dc20-7565-428d-b70b-92f72cc22f0e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:18.654195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322dc20-7565-428d-b70b-92f72cc22f0e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:18.654195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322dc20-7565-428d-b70b-92f72cc22f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:18.690794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-5322dc20-7565-428d-b70b-92f72cc22f0e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:18.692032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5322dc20-7565-428d-b70b-92f72cc22f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:18.865419 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb1e58-aed9-4031-9c18-4b7ca6a344da None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:18.866014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb1e58-aed9-4031-9c18-4b7ca6a344da None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:18.866551 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb1e58-aed9-4031-9c18-4b7ca6a344da None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:18.868552 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 932/1843] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:46:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:18.877140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 912/1844] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:18.915422 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322dc20-7565-428d-b70b-92f72cc22f0e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:18.915642 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe584d8-ee5a-44f5-991b-c618259f1182 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:18.915884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe584d8-ee5a-44f5-991b-c618259f1182 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:18.916259 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe584d8-ee5a-44f5-991b-c618259f1182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:18.917930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5322dc20-7565-428d-b70b-92f72cc22f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:18.953406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-0fe584d8-ee5a-44f5-991b-c618259f1182 None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:18.962282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-0fe584d8-ee5a-44f5-991b-c618259f1182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:19.032341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:19.089253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322dc20-7565-428d-b70b-92f72cc22f0e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:19.098423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fe34c-60b7-4e33-8ef8-f1be0a7ffdf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:19.099826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fe34c-60b7-4e33-8ef8-f1be0a7ffdf3 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:19.100747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fe34c-60b7-4e33-8ef8-f1be0a7ffdf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:19.101623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9fe34c-60b7-4e33-8ef8-f1be0a7ffdf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:19.119147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9fe34c-60b7-4e33-8ef8-f1be0a7ffdf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:19.119147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9fe34c-60b7-4e33-8ef8-f1be0a7ffdf3 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:19.156394 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 913/1845] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:19.159706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 933/1846] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:46:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:19.410242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fe34c-60b7-4e33-8ef8-f1be0a7ffdf3 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:19.411967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1f9fe34c-60b7-4e33-8ef8-f1be0a7ffdf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:19.434618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9fe34c-60b7-4e33-8ef8-f1be0a7ffdf3 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:19.441867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe1838-9de6-4bfb-b64f-a2af1abf407a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:19.442613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe1838-9de6-4bfb-b64f-a2af1abf407a service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:19.443051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe1838-9de6-4bfb-b64f-a2af1abf407a service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:19.443498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe1838-9de6-4bfb-b64f-a2af1abf407a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:19.454128 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe1838-9de6-4bfb-b64f-a2af1abf407a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:19.455891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe1838-9de6-4bfb-b64f-a2af1abf407a service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:19.468457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 914/1847] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:19.886990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe584d8-ee5a-44f5-991b-c618259f1182 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:19.888715 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-0fe584d8-ee5a-44f5-991b-c618259f1182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:19.908863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe584d8-ee5a-44f5-991b-c618259f1182 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:19.913475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:19.914811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:19.915097 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:19.915373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] PATH_INFO: `/v3/projects/cb03cd19940a4860b827dc30d3db8fc8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:19.916550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb03cd19940a4860b827dc30d3db8fc8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:19.918949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:20.021805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb03cd19940a4860b827dc30d3db8fc8', 'request_id': 'req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:20.029850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb03cd19940a4860b827dc30d3db8fc8', 'request_id': 'req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:20.037069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb03cd19940a4860b827dc30d3db8fc8', 'request_id': 'req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:20.065787 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] The token cache is being invalidate because project cb03cd19940a4860b827dc30d3db8fc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:20.066918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:20.105060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:20.110604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 934/1848] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:19 2022] DELETE /identity/v3/projects/cb03cd19940a4860b827dc30d3db8fc8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:20.116437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe1838-9de6-4bfb-b64f-a2af1abf407a service nova] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:20.205485 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe1838-9de6-4bfb-b64f-a2af1abf407a service nova] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:20.430712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052358d-bb8a-4427-981f-d41f68c8997c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:20.431452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052358d-bb8a-4427-981f-d41f68c8997c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:20.431881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052358d-bb8a-4427-981f-d41f68c8997c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:20.432283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052358d-bb8a-4427-981f-d41f68c8997c admin admin] PATH_INFO: `/v3/users/3b23aed057db43feb9ca8f6db985f1b9` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:20.434015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052358d-bb8a-4427-981f-d41f68c8997c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b23aed057db43feb9ca8f6db985f1b9)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:20.438223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052358d-bb8a-4427-981f-d41f68c8997c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:20.534538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4052358d-bb8a-4427-981f-d41f68c8997c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b23aed057db43feb9ca8f6db985f1b9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:20.535243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4052358d-bb8a-4427-981f-d41f68c8997c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b23aed057db43feb9ca8f6db985f1b9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:20.550098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4052358d-bb8a-4427-981f-d41f68c8997c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b23aed057db43feb9ca8f6db985f1b9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:20.557947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4052358d-bb8a-4427-981f-d41f68c8997c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b23aed057db43feb9ca8f6db985f1b9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:20.558725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4052358d-bb8a-4427-981f-d41f68c8997c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b23aed057db43feb9ca8f6db985f1b9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:20.578957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 915/1849] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:20 2022] DELETE /identity/v3/users/3b23aed057db43feb9ca8f6db985f1b9 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:21.507636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:21.510135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:21.533909 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c572378-f3d0-4efb-b36d-9fb3b2db8dfc admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:21.670752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:21.671457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:21.671749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:21.672026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0 admin admin] PATH_INFO: `/v3/projects/8ab473d955a34e2494f6897d2948e61a` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:21.673187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab473d955a34e2494f6897d2948e61a)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:21.675620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:21.696439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:21.752892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab473d955a34e2494f6897d2948e61a', 'request_id': 'req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:21.767147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab473d955a34e2494f6897d2948e61a', 'request_id': 'req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:21.777095 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab473d955a34e2494f6897d2948e61a', 'request_id': 'req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:21.799070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0 admin admin] The token cache is being invalidate because project 8ab473d955a34e2494f6897d2948e61a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:21.799605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:21.800339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7115d535-b34f-4d6a-a5eb-591a0f1f37f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:21.802107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 935/1850] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:21 2022] DELETE /identity/v3/projects/8ab473d955a34e2494f6897d2948e61a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:21.807634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4800bf1-69b2-483a-a98f-7647404ca968 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:21.808180 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4800bf1-69b2-483a-a98f-7647404ca968 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:21.808509 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4800bf1-69b2-483a-a98f-7647404ca968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:21.843165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d4800bf1-69b2-483a-a98f-7647404ca968 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:21.844424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d4800bf1-69b2-483a-a98f-7647404ca968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:21.896882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:21.938389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 916/1851] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:21.946464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7a48c-25fa-47b2-b805-a549485ebefc None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:21.946779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7a48c-25fa-47b2-b805-a549485ebefc None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:21.947060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7a48c-25fa-47b2-b805-a549485ebefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:21.972863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-fce7a48c-25fa-47b2-b805-a549485ebefc None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:21.981457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fce7a48c-25fa-47b2-b805-a549485ebefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:22.003231 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 936/1852] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:22.007440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4800bf1-69b2-483a-a98f-7647404ca968 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.017612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4800bf1-69b2-483a-a98f-7647404ca968 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.025432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b15c5-022c-4299-a754-a9490ceec53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.027410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b15c5-022c-4299-a754-a9490ceec53e admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.027813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b15c5-022c-4299-a754-a9490ceec53e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.028198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b15c5-022c-4299-a754-a9490ceec53e admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.028972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b15c5-022c-4299-a754-a9490ceec53e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:22.029585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b15c5-022c-4299-a754-a9490ceec53e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.031164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b15c5-022c-4299-a754-a9490ceec53e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.047379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 917/1853] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:46:22.051882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7a48c-25fa-47b2-b805-a549485ebefc None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.062459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7a48c-25fa-47b2-b805-a549485ebefc None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.070508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074c4e7-5bc3-4ebb-9d72-fd0e77fd4df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.071134 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074c4e7-5bc3-4ebb-9d72-fd0e77fd4df7 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.071409 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074c4e7-5bc3-4ebb-9d72-fd0e77fd4df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.071679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074c4e7-5bc3-4ebb-9d72-fd0e77fd4df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.072357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074c4e7-5bc3-4ebb-9d72-fd0e77fd4df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.074410 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074c4e7-5bc3-4ebb-9d72-fd0e77fd4df7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.076543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500cecb-f6a2-4325-b998-bde95d488ecb None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.077132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500cecb-f6a2-4325-b998-bde95d488ecb None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.077419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500cecb-f6a2-4325-b998-bde95d488ecb None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.079734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 918/1854] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:46:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:46:22.096278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c128-315f-4693-a5a0-950cf01c316e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.096278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c128-315f-4693-a5a0-950cf01c316e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.096547 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7c128-315f-4693-a5a0-950cf01c316e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.116437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 937/1855] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:22 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:22.120474 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-57e7c128-315f-4693-a5a0-950cf01c316e None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:22.120840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-57e7c128-315f-4693-a5a0-950cf01c316e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:22.121502 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074c4e7-5bc3-4ebb-9d72-fd0e77fd4df7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.186289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074c4e7-5bc3-4ebb-9d72-fd0e77fd4df7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.189799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 919/1856] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:46:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:22.190938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d7bd4-5ec4-4d1e-87ac-27c6d45adcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.191974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d7bd4-5ec4-4d1e-87ac-27c6d45adcd3 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.192252 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d7bd4-5ec4-4d1e-87ac-27c6d45adcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.192529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d7bd4-5ec4-4d1e-87ac-27c6d45adcd3 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.194509 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d7bd4-5ec4-4d1e-87ac-27c6d45adcd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.195655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d7bd4-5ec4-4d1e-87ac-27c6d45adcd3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.202888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7c128-315f-4693-a5a0-950cf01c316e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.204771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-57e7c128-315f-4693-a5a0-950cf01c316e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:22.222332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7c128-315f-4693-a5a0-950cf01c316e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.224588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 938/1857] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:22 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:22.230054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082dd30-c876-43ca-a0ea-2c55f52bff3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.230661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d7bd4-5ec4-4d1e-87ac-27c6d45adcd3 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.230784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082dd30-c876-43ca-a0ea-2c55f52bff3f service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.230939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082dd30-c876-43ca-a0ea-2c55f52bff3f service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.231327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082dd30-c876-43ca-a0ea-2c55f52bff3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.242984 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082dd30-c876-43ca-a0ea-2c55f52bff3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.244626 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082dd30-c876-43ca-a0ea-2c55f52bff3f service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.247991 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d7bd4-5ec4-4d1e-87ac-27c6d45adcd3 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.252997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445c12d-f999-4867-8ac4-d071413b306c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.253672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445c12d-f999-4867-8ac4-d071413b306c admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.253922 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445c12d-f999-4867-8ac4-d071413b306c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.254242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445c12d-f999-4867-8ac4-d071413b306c admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.254899 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445c12d-f999-4867-8ac4-d071413b306c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.256747 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445c12d-f999-4867-8ac4-d071413b306c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.256875 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 920/1858] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:22.267672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 939/1859] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:22.274708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082dd30-c876-43ca-a0ea-2c55f52bff3f service nova] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.286520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082dd30-c876-43ca-a0ea-2c55f52bff3f service nova] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.297633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d9f8d-3f2a-463a-a8d9-2f5c1aba8b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.298397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d9f8d-3f2a-463a-a8d9-2f5c1aba8b44 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.298397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d9f8d-3f2a-463a-a8d9-2f5c1aba8b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.298802 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d9f8d-3f2a-463a-a8d9-2f5c1aba8b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.299319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d9f8d-3f2a-463a-a8d9-2f5c1aba8b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.300865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d9f8d-3f2a-463a-a8d9-2f5c1aba8b44 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.311551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 921/1860] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:22.315956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445c12d-f999-4867-8ac4-d071413b306c admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.326370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445c12d-f999-4867-8ac4-d071413b306c admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.332563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993f7f1-490b-4a06-935f-e817af2361d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.333326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993f7f1-490b-4a06-935f-e817af2361d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.333610 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993f7f1-490b-4a06-935f-e817af2361d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.333924 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993f7f1-490b-4a06-935f-e817af2361d6 admin admin] PATH_INFO: `/v3/projects/f0cd1f7ca77641749aa42adee5ca7107/users/72dd3e9f71d34978bbb29c1210d45ee2/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.352899 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993f7f1-490b-4a06-935f-e817af2361d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0cd1f7ca77641749aa42adee5ca7107, user_id=72dd3e9f71d34978bbb29c1210d45ee2, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.355688 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993f7f1-490b-4a06-935f-e817af2361d6 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.370109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 940/1861] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:22 2022] PUT /identity/v3/projects/f0cd1f7ca77641749aa42adee5ca7107/users/72dd3e9f71d34978bbb29c1210d45ee2/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:22.381159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c326e2a-2bbb-4223-aa0e-d7a20217aab8 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.381453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c326e2a-2bbb-4223-aa0e-d7a20217aab8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.381761 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c326e2a-2bbb-4223-aa0e-d7a20217aab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.410607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-0c326e2a-2bbb-4223-aa0e-d7a20217aab8 None None] MFA Rules not processed for user `72dd3e9f71d34978bbb29c1210d45ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:22.411852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0c326e2a-2bbb-4223-aa0e-d7a20217aab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:22.478066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 922/1862] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:22.482612 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993f7f1-490b-4a06-935f-e817af2361d6 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.513034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:22.547158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993f7f1-490b-4a06-935f-e817af2361d6 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.551763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc46a4b-ac27-4727-bec1-9125befb15ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.552364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc46a4b-ac27-4727-bec1-9125befb15ba admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.552643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc46a4b-ac27-4727-bec1-9125befb15ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.552916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc46a4b-ac27-4727-bec1-9125befb15ba admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.553537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc46a4b-ac27-4727-bec1-9125befb15ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.555514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc46a4b-ac27-4727-bec1-9125befb15ba admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.580462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 941/1863] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:22 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:22.584454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c326e2a-2bbb-4223-aa0e-d7a20217aab8 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.596030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c326e2a-2bbb-4223-aa0e-d7a20217aab8 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.616098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:22.661132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37be6b0-ad02-4fd4-b34b-2221b95eb020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.661785 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37be6b0-ad02-4fd4-b34b-2221b95eb020 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.662068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37be6b0-ad02-4fd4-b34b-2221b95eb020 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.662342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37be6b0-ad02-4fd4-b34b-2221b95eb020 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.662986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37be6b0-ad02-4fd4-b34b-2221b95eb020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.665575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37be6b0-ad02-4fd4-b34b-2221b95eb020 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.686885 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 923/1864] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:22 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:22.690901 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc46a4b-ac27-4727-bec1-9125befb15ba admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.701782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc46a4b-ac27-4727-bec1-9125befb15ba admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.706360 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a488f16-76d5-454a-838d-551b5a7ae46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.706954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a488f16-76d5-454a-838d-551b5a7ae46b admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.707232 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a488f16-76d5-454a-838d-551b5a7ae46b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.707518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a488f16-76d5-454a-838d-551b5a7ae46b admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.708139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a488f16-76d5-454a-838d-551b5a7ae46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.709789 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a488f16-76d5-454a-838d-551b5a7ae46b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.723901 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 942/1865] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:22.724989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37be6b0-ad02-4fd4-b34b-2221b95eb020 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.734852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37be6b0-ad02-4fd4-b34b-2221b95eb020 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.739562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939eff7e-97c6-4dce-9bd4-9aa574b7e471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.740215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939eff7e-97c6-4dce-9bd4-9aa574b7e471 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.740493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939eff7e-97c6-4dce-9bd4-9aa574b7e471 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.740766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939eff7e-97c6-4dce-9bd4-9aa574b7e471 admin admin] PATH_INFO: `/v3/projects/d4485181dd6448fa8c49df0dbbb9808e/users/30e4df44f7f84076929beec1e9a4f084/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.755583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939eff7e-97c6-4dce-9bd4-9aa574b7e471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4485181dd6448fa8c49df0dbbb9808e, user_id=30e4df44f7f84076929beec1e9a4f084, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.758788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939eff7e-97c6-4dce-9bd4-9aa574b7e471 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.772996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 924/1866] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:22 2022] PUT /identity/v3/projects/d4485181dd6448fa8c49df0dbbb9808e/users/30e4df44f7f84076929beec1e9a4f084/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:22.777587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a488f16-76d5-454a-838d-551b5a7ae46b admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.787984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a488f16-76d5-454a-838d-551b5a7ae46b admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.843882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24057a69-8373-4cb1-93a3-8a1dd9679719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.844482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24057a69-8373-4cb1-93a3-8a1dd9679719 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.844757 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24057a69-8373-4cb1-93a3-8a1dd9679719 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.845286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24057a69-8373-4cb1-93a3-8a1dd9679719 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.845950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24057a69-8373-4cb1-93a3-8a1dd9679719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.847535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24057a69-8373-4cb1-93a3-8a1dd9679719 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.857022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 943/1867] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:22.861223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939eff7e-97c6-4dce-9bd4-9aa574b7e471 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:22.871984 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939eff7e-97c6-4dce-9bd4-9aa574b7e471 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:22.876924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c66be-6269-4bc4-989d-a035efb156c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:22.877675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c66be-6269-4bc4-989d-a035efb156c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.878077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c66be-6269-4bc4-989d-a035efb156c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.878418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c66be-6269-4bc4-989d-a035efb156c7 admin admin] PATH_INFO: `/v3/projects/d4485181dd6448fa8c49df0dbbb9808e/users/30e4df44f7f84076929beec1e9a4f084/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.880522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c66be-6269-4bc4-989d-a035efb156c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4485181dd6448fa8c49df0dbbb9808e, user_id=30e4df44f7f84076929beec1e9a4f084, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:22.883924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c66be-6269-4bc4-989d-a035efb156c7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:22.895910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 925/1868] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:22 2022] PUT /identity/v3/projects/d4485181dd6448fa8c49df0dbbb9808e/users/30e4df44f7f84076929beec1e9a4f084/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:22.907479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3d6f2-97ac-4a43-97b7-c0cb378c0329 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:22.907882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3d6f2-97ac-4a43-97b7-c0cb378c0329 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:22.908161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3d6f2-97ac-4a43-97b7-c0cb378c0329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:22.940452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-7ac3d6f2-97ac-4a43-97b7-c0cb378c0329 None None] MFA Rules not processed for user `30e4df44f7f84076929beec1e9a4f084`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:22.941732 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7ac3d6f2-97ac-4a43-97b7-c0cb378c0329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:23.019303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 944/1869] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:23.032634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c66be-6269-4bc4-989d-a035efb156c7 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:23.035382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-700c66be-6269-4bc4-989d-a035efb156c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:23.097089 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c66be-6269-4bc4-989d-a035efb156c7 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:23.100888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec59a55-5184-4595-9a78-7c6c6111c7ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:23.101520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec59a55-5184-4595-9a78-7c6c6111c7ca service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:23.101831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec59a55-5184-4595-9a78-7c6c6111c7ca service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:23.102105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec59a55-5184-4595-9a78-7c6c6111c7ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:23.111353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec59a55-5184-4595-9a78-7c6c6111c7ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:23.112775 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec59a55-5184-4595-9a78-7c6c6111c7ca service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:23.122967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 926/1870] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:23.209782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3d6f2-97ac-4a43-97b7-c0cb378c0329 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:23.219103 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3d6f2-97ac-4a43-97b7-c0cb378c0329 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:23.222880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb905a8-e13f-4c5b-8593-49ceb65dd2e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:23.223473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb905a8-e13f-4c5b-8593-49ceb65dd2e9 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:23.223750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb905a8-e13f-4c5b-8593-49ceb65dd2e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:23.224019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb905a8-e13f-4c5b-8593-49ceb65dd2e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:23.242918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:23.282436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb905a8-e13f-4c5b-8593-49ceb65dd2e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:23.283814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb905a8-e13f-4c5b-8593-49ceb65dd2e9 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:23.292743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 945/1871] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:23.601953 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec59a55-5184-4595-9a78-7c6c6111c7ca service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:23.724948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec59a55-5184-4595-9a78-7c6c6111c7ca service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:23.730863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:23.731633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:23.732015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:23.732386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] PATH_INFO: `/v3/users/72dd3e9f71d34978bbb29c1210d45ee2` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:23.734549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72dd3e9f71d34978bbb29c1210d45ee2)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:23.745830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:23.767999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:23.794651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72dd3e9f71d34978bbb29c1210d45ee2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:23.795298 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72dd3e9f71d34978bbb29c1210d45ee2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:23.807446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72dd3e9f71d34978bbb29c1210d45ee2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:23.813990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72dd3e9f71d34978bbb29c1210d45ee2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:23.814620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72dd3e9f71d34978bbb29c1210d45ee2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:23.828918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 927/1872] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:23 2022] DELETE /identity/v3/users/72dd3e9f71d34978bbb29c1210d45ee2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:23.833294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b23f3-c7e4-44f0-baea-e81f28628977 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:23.833681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b23f3-c7e4-44f0-baea-e81f28628977 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:23.833873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b23f3-c7e4-44f0-baea-e81f28628977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:23.852626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-7d3b23f3-c7e4-44f0-baea-e81f28628977 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:23.853862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7d3b23f3-c7e4-44f0-baea-e81f28628977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:23.913768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 946/1873] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:23.923481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:23.924894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:23.982402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5be62-a2aa-4a62-86d4-abacc785b8e7 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:23.985994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67bdaf-a9dc-4062-b0d4-274bdc648033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:23.986600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67bdaf-a9dc-4062-b0d4-274bdc648033 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:23.986879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67bdaf-a9dc-4062-b0d4-274bdc648033 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:23.987149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67bdaf-a9dc-4062-b0d4-274bdc648033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:23.996435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67bdaf-a9dc-4062-b0d4-274bdc648033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:23.997892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67bdaf-a9dc-4062-b0d4-274bdc648033 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:24.007133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 928/1874] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:24.340913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b23f3-c7e4-44f0-baea-e81f28628977 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:24.342425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7d3b23f3-c7e4-44f0-baea-e81f28628977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:24.354932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b23f3-c7e4-44f0-baea-e81f28628977 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:24.359630 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:24.360299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:24.360581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:24.360856 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] PATH_INFO: `/v3/projects/f0cd1f7ca77641749aa42adee5ca7107` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:24.362020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0cd1f7ca77641749aa42adee5ca7107)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:24.363944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:24.412277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0cd1f7ca77641749aa42adee5ca7107', 'request_id': 'req-7f0a237a-73be-4f65-9a23-187c9b2605e9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:24.418820 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0cd1f7ca77641749aa42adee5ca7107', 'request_id': 'req-7f0a237a-73be-4f65-9a23-187c9b2605e9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:24.424277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0cd1f7ca77641749aa42adee5ca7107', 'request_id': 'req-7f0a237a-73be-4f65-9a23-187c9b2605e9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:24.436852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] The token cache is being invalidate because project f0cd1f7ca77641749aa42adee5ca7107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:24.437144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:24.437804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:24.439304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 947/1875] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:24 2022] DELETE /identity/v3/projects/f0cd1f7ca77641749aa42adee5ca7107 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:24.443344 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67bdaf-a9dc-4062-b0d4-274bdc648033 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:24.453038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67bdaf-a9dc-4062-b0d4-274bdc648033 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:24.509521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508d36b-2c92-4fed-b0f4-40ac93c6f761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:24.510176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d36b-2c92-4fed-b0f4-40ac93c6f761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:24.510460 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d36b-2c92-4fed-b0f4-40ac93c6f761 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:24.510737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508d36b-2c92-4fed-b0f4-40ac93c6f761 admin admin] PATH_INFO: `/v3/users/30e4df44f7f84076929beec1e9a4f084` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:24.511862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508d36b-2c92-4fed-b0f4-40ac93c6f761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30e4df44f7f84076929beec1e9a4f084)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:24.513816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508d36b-2c92-4fed-b0f4-40ac93c6f761 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:24.555675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:24.558204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5508d36b-2c92-4fed-b0f4-40ac93c6f761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30e4df44f7f84076929beec1e9a4f084'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:24.559326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5508d36b-2c92-4fed-b0f4-40ac93c6f761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30e4df44f7f84076929beec1e9a4f084'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:24.571617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5508d36b-2c92-4fed-b0f4-40ac93c6f761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30e4df44f7f84076929beec1e9a4f084'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:24.578289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5508d36b-2c92-4fed-b0f4-40ac93c6f761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30e4df44f7f84076929beec1e9a4f084'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:24.578925 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5508d36b-2c92-4fed-b0f4-40ac93c6f761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30e4df44f7f84076929beec1e9a4f084'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:24.591454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 929/1876] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:24 2022] DELETE /identity/v3/users/30e4df44f7f84076929beec1e9a4f084 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:24.916263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:24.917815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:24.947181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a237a-73be-4f65-9a23-187c9b2605e9 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:24.998591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:25.015821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:25.016455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.016736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.017011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3 admin admin] PATH_INFO: `/v3/projects/d4485181dd6448fa8c49df0dbbb9808e` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.018164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4485181dd6448fa8c49df0dbbb9808e)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:25.020070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:25.067711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4485181dd6448fa8c49df0dbbb9808e', 'request_id': 'req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:25.073866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4485181dd6448fa8c49df0dbbb9808e', 'request_id': 'req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:25.079380 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4485181dd6448fa8c49df0dbbb9808e', 'request_id': 'req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:25.092174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3 admin admin] The token cache is being invalidate because project d4485181dd6448fa8c49df0dbbb9808e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:25.092469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:25.093091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7e84f0ec-e112-4401-9dcd-ec57f791e4c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:25.094623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 948/1877] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:24 2022] DELETE /identity/v3/projects/d4485181dd6448fa8c49df0dbbb9808e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:25.100861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22781372-7d1e-4847-b02e-a64ccad2e1ae None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.101903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22781372-7d1e-4847-b02e-a64ccad2e1ae None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.102005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22781372-7d1e-4847-b02e-a64ccad2e1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.121767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-22781372-7d1e-4847-b02e-a64ccad2e1ae None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:25.122925 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-22781372-7d1e-4847-b02e-a64ccad2e1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:25.186763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 930/1878] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:25.194637 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d4f3d-63e3-45c2-8a53-bf153a97fff7 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.194992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d4f3d-63e3-45c2-8a53-bf153a97fff7 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.195278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d4f3d-63e3-45c2-8a53-bf153a97fff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.216022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-4a4d4f3d-63e3-45c2-8a53-bf153a97fff7 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:25.217224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4a4d4f3d-63e3-45c2-8a53-bf153a97fff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:25.227854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 949/1879] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:25.232094 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22781372-7d1e-4847-b02e-a64ccad2e1ae None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:25.243468 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22781372-7d1e-4847-b02e-a64ccad2e1ae None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:25.248162 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9b49a-dc43-4f57-b1a9-8ec98ecc8e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:25.248844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9b49a-dc43-4f57-b1a9-8ec98ecc8e53 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.249189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9b49a-dc43-4f57-b1a9-8ec98ecc8e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.249537 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9b49a-dc43-4f57-b1a9-8ec98ecc8e53 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.250291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9b49a-dc43-4f57-b1a9-8ec98ecc8e53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:25.250655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9b49a-dc43-4f57-b1a9-8ec98ecc8e53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:25.252272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9b49a-dc43-4f57-b1a9-8ec98ecc8e53 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:25.266546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 931/1880] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:46:25.270946 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d4f3d-63e3-45c2-8a53-bf153a97fff7 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:25.280787 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d4f3d-63e3-45c2-8a53-bf153a97fff7 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:25.285444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3b4f9-b917-4139-b74c-c552088f62f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:25.286077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3b4f9-b917-4139-b74c-c552088f62f7 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.286357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3b4f9-b917-4139-b74c-c552088f62f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.286629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3b4f9-b917-4139-b74c-c552088f62f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.287258 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3b4f9-b917-4139-b74c-c552088f62f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:25.290051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3b4f9-b917-4139-b74c-c552088f62f7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:25.320487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 950/1881] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:25 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:25.324569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9b49a-dc43-4f57-b1a9-8ec98ecc8e53 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:25.335941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9b49a-dc43-4f57-b1a9-8ec98ecc8e53 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:25.392242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfa4d7-7817-4e73-8568-ed9022ae9f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:25.392816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfa4d7-7817-4e73-8568-ed9022ae9f4a admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.393099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfa4d7-7817-4e73-8568-ed9022ae9f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.393374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfa4d7-7817-4e73-8568-ed9022ae9f4a admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.394042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfa4d7-7817-4e73-8568-ed9022ae9f4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:25.395956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfa4d7-7817-4e73-8568-ed9022ae9f4a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:25.421993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 932/1882] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:25 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:25.427282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3b4f9-b917-4139-b74c-c552088f62f7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:25.440285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3b4f9-b917-4139-b74c-c552088f62f7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:25.445368 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a6d3b-8c71-4124-9d2d-1400c41c0cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:25.446023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a6d3b-8c71-4124-9d2d-1400c41c0cbd admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.446300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a6d3b-8c71-4124-9d2d-1400c41c0cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.446571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a6d3b-8c71-4124-9d2d-1400c41c0cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.447213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a6d3b-8c71-4124-9d2d-1400c41c0cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:25.449683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a6d3b-8c71-4124-9d2d-1400c41c0cbd admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:25.465945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 951/1883] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:46:25.467098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfa4d7-7817-4e73-8568-ed9022ae9f4a admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:25.478501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfa4d7-7817-4e73-8568-ed9022ae9f4a admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:25.483968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856d921-3838-44ae-af77-7a852d8132c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:25.484688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856d921-3838-44ae-af77-7a852d8132c4 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.485265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856d921-3838-44ae-af77-7a852d8132c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.485399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856d921-3838-44ae-af77-7a852d8132c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.486318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856d921-3838-44ae-af77-7a852d8132c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:25.488128 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856d921-3838-44ae-af77-7a852d8132c4 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:25.503400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 933/1884] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:25.507838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a6d3b-8c71-4124-9d2d-1400c41c0cbd admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:25.523865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a6d3b-8c71-4124-9d2d-1400c41c0cbd admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:25.531291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f549a6-28c6-4810-9d98-df37210eb048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:25.532193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f549a6-28c6-4810-9d98-df37210eb048 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.532546 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f549a6-28c6-4810-9d98-df37210eb048 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.532897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f549a6-28c6-4810-9d98-df37210eb048 admin admin] PATH_INFO: `/v3/projects/547e5b4cc87d434fb938eb6409ea1f34/users/ed746a24940a4042b04d5fee3b1353df/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.554288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f549a6-28c6-4810-9d98-df37210eb048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=547e5b4cc87d434fb938eb6409ea1f34, user_id=ed746a24940a4042b04d5fee3b1353df, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:25.556977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f549a6-28c6-4810-9d98-df37210eb048 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:25.582121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 952/1885] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:25 2022] PUT /identity/v3/projects/547e5b4cc87d434fb938eb6409ea1f34/users/ed746a24940a4042b04d5fee3b1353df/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:25.588222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9857a-269a-4f14-be3e-d85b09af9347 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.588565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9857a-269a-4f14-be3e-d85b09af9347 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.589016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9857a-269a-4f14-be3e-d85b09af9347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.625090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-11e9857a-269a-4f14-be3e-d85b09af9347 None None] MFA Rules not processed for user `ed746a24940a4042b04d5fee3b1353df`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:25.626468 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-11e9857a-269a-4f14-be3e-d85b09af9347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:25.643638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:25.722520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 934/1886] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:25.727021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f549a6-28c6-4810-9d98-df37210eb048 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:25.789438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f549a6-28c6-4810-9d98-df37210eb048 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:25.794044 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7cef3-3a71-4414-99f8-a2c539bdb179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:25.794763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7cef3-3a71-4414-99f8-a2c539bdb179 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.795047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7cef3-3a71-4414-99f8-a2c539bdb179 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.795320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7cef3-3a71-4414-99f8-a2c539bdb179 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.795963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7cef3-3a71-4414-99f8-a2c539bdb179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:25.797889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7cef3-3a71-4414-99f8-a2c539bdb179 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:25.812769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:25.830497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 953/1887] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:25 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:25.832014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9857a-269a-4f14-be3e-d85b09af9347 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:25.843620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9857a-269a-4f14-be3e-d85b09af9347 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:25.914536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d455a-d6c1-4037-9934-598459af29d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:25.916032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d455a-d6c1-4037-9934-598459af29d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.916032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d455a-d6c1-4037-9934-598459af29d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.916323 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d455a-d6c1-4037-9934-598459af29d7 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.917087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d455a-d6c1-4037-9934-598459af29d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:25.919421 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d455a-d6c1-4037-9934-598459af29d7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:25.946485 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 935/1888] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:25 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:25.950345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7cef3-3a71-4414-99f8-a2c539bdb179 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:25.968489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7cef3-3a71-4414-99f8-a2c539bdb179 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:25.973696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f90a0-84bb-4cbc-8b6c-0fa548432c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:25.974339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f90a0-84bb-4cbc-8b6c-0fa548432c23 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:25.974624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f90a0-84bb-4cbc-8b6c-0fa548432c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:25.974901 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f90a0-84bb-4cbc-8b6c-0fa548432c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:25.975568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f90a0-84bb-4cbc-8b6c-0fa548432c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:25.977873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f90a0-84bb-4cbc-8b6c-0fa548432c23 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:25.992584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 954/1889] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:46:25.993782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d455a-d6c1-4037-9934-598459af29d7 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:26.004650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d455a-d6c1-4037-9934-598459af29d7 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:26.011478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed42f0-1d1a-4f40-90dd-7aa351161db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:26.012156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed42f0-1d1a-4f40-90dd-7aa351161db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:26.012441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed42f0-1d1a-4f40-90dd-7aa351161db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:26.012718 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed42f0-1d1a-4f40-90dd-7aa351161db5 admin admin] PATH_INFO: `/v3/projects/0c9336a58cdc4ab795ee01190c9fdc42/users/bc89839ffc8e47d19d19b36de72fd290/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:26.038173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed42f0-1d1a-4f40-90dd-7aa351161db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9336a58cdc4ab795ee01190c9fdc42, user_id=bc89839ffc8e47d19d19b36de72fd290, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:26.044792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed42f0-1d1a-4f40-90dd-7aa351161db5 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:26.061597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 936/1890] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:25 2022] PUT /identity/v3/projects/0c9336a58cdc4ab795ee01190c9fdc42/users/bc89839ffc8e47d19d19b36de72fd290/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:26.062881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f90a0-84bb-4cbc-8b6c-0fa548432c23 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:26.074395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f90a0-84bb-4cbc-8b6c-0fa548432c23 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:26.137496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cefd75-de26-4f79-89ce-8f1d7a06fac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:26.137900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cefd75-de26-4f79-89ce-8f1d7a06fac7 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:26.138527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cefd75-de26-4f79-89ce-8f1d7a06fac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:26.138880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cefd75-de26-4f79-89ce-8f1d7a06fac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:26.139995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cefd75-de26-4f79-89ce-8f1d7a06fac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:26.141493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cefd75-de26-4f79-89ce-8f1d7a06fac7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:26.152731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 955/1891] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:26.166117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed42f0-1d1a-4f40-90dd-7aa351161db5 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:26.176756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed42f0-1d1a-4f40-90dd-7aa351161db5 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:26.193941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae7c08-346f-42f2-b328-7a6e274382e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:26.199507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7c08-346f-42f2-b328-7a6e274382e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:26.200239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7c08-346f-42f2-b328-7a6e274382e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:26.200689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae7c08-346f-42f2-b328-7a6e274382e8 admin admin] PATH_INFO: `/v3/projects/0c9336a58cdc4ab795ee01190c9fdc42/users/bc89839ffc8e47d19d19b36de72fd290/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:26.203717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae7c08-346f-42f2-b328-7a6e274382e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9336a58cdc4ab795ee01190c9fdc42, user_id=bc89839ffc8e47d19d19b36de72fd290, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:26.208994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae7c08-346f-42f2-b328-7a6e274382e8 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:26.223104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 937/1892] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:26 2022] PUT /identity/v3/projects/0c9336a58cdc4ab795ee01190c9fdc42/users/bc89839ffc8e47d19d19b36de72fd290/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:26.234127 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ffcac-bbe5-4d83-8548-5339d86f9fe2 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:26.234419 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ffcac-bbe5-4d83-8548-5339d86f9fe2 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:26.234705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ffcac-bbe5-4d83-8548-5339d86f9fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:26.267598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-762ffcac-bbe5-4d83-8548-5339d86f9fe2 None None] MFA Rules not processed for user `bc89839ffc8e47d19d19b36de72fd290`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:26.269032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-762ffcac-bbe5-4d83-8548-5339d86f9fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:26.359841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 956/1893] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:26.370707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae7c08-346f-42f2-b328-7a6e274382e8 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:26.372510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b0ae7c08-346f-42f2-b328-7a6e274382e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:26.429668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae7c08-346f-42f2-b328-7a6e274382e8 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:26.433311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec93169-2f59-404f-8ca4-f491ca5c51a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:26.433944 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec93169-2f59-404f-8ca4-f491ca5c51a1 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:26.434226 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec93169-2f59-404f-8ca4-f491ca5c51a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:26.434498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec93169-2f59-404f-8ca4-f491ca5c51a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:26.455728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:26.495859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec93169-2f59-404f-8ca4-f491ca5c51a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:26.497463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec93169-2f59-404f-8ca4-f491ca5c51a1 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:26.509998 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 938/1894] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:31.154501 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ffcac-bbe5-4d83-8548-5339d86f9fe2 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:31.165883 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ffcac-bbe5-4d83-8548-5339d86f9fe2 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:31.177144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94b4c4-281f-4c31-a880-09ccb70adb25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:31.178310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94b4c4-281f-4c31-a880-09ccb70adb25 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:31.178623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94b4c4-281f-4c31-a880-09ccb70adb25 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:31.178904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94b4c4-281f-4c31-a880-09ccb70adb25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:31.193984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94b4c4-281f-4c31-a880-09ccb70adb25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:31.196013 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94b4c4-281f-4c31-a880-09ccb70adb25 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:31.209498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 957/1895] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:31.258707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec93169-2f59-404f-8ca4-f491ca5c51a1 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:31.260262 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8ec93169-2f59-404f-8ca4-f491ca5c51a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:31.361254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec93169-2f59-404f-8ca4-f491ca5c51a1 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:31.367374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:31.368530 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:31.368932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:31.369880 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] PATH_INFO: `/v3/users/44d19bdc64da4d72ab5e37aa53188b5c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:31.371714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44d19bdc64da4d72ab5e37aa53188b5c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:31.374329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:31.478164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44d19bdc64da4d72ab5e37aa53188b5c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:31.479327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44d19bdc64da4d72ab5e37aa53188b5c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:31.495603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44d19bdc64da4d72ab5e37aa53188b5c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:31.507420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44d19bdc64da4d72ab5e37aa53188b5c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:31.509879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44d19bdc64da4d72ab5e37aa53188b5c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:31.532371 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 939/1896] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:31 2022] DELETE /identity/v3/users/44d19bdc64da4d72ab5e37aa53188b5c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:31.536714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771a28d-efb9-4757-b7c2-944ab5098d81 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:31.537004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771a28d-efb9-4757-b7c2-944ab5098d81 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:31.537281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771a28d-efb9-4757-b7c2-944ab5098d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:31.564585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-7771a28d-efb9-4757-b7c2-944ab5098d81 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:31.566139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7771a28d-efb9-4757-b7c2-944ab5098d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:31.675718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:31.703498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 958/1897] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:31.713754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:31.717376 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:31.801786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:31.820951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5dd937-7969-41cd-ae64-736b9941e657 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:31.825452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64c2d7-e897-451f-b264-fc7b55c08cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:31.826117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64c2d7-e897-451f-b264-fc7b55c08cbd service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:31.826686 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64c2d7-e897-451f-b264-fc7b55c08cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:31.827115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64c2d7-e897-451f-b264-fc7b55c08cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:31.840700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64c2d7-e897-451f-b264-fc7b55c08cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:31.842701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64c2d7-e897-451f-b264-fc7b55c08cbd service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:31.859087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 940/1898] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:32.228912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771a28d-efb9-4757-b7c2-944ab5098d81 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:32.230494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7771a28d-efb9-4757-b7c2-944ab5098d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:32.246333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771a28d-efb9-4757-b7c2-944ab5098d81 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:32.252276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:32.253145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:32.253571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:32.254032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a admin admin] PATH_INFO: `/v3/projects/fcafb472d06f4b5eb1ca07e95611b88a` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:32.255362 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcafb472d06f4b5eb1ca07e95611b88a)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:32.258761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:32.320832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcafb472d06f4b5eb1ca07e95611b88a', 'request_id': 'req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:32.329779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcafb472d06f4b5eb1ca07e95611b88a', 'request_id': 'req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:32.338082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcafb472d06f4b5eb1ca07e95611b88a', 'request_id': 'req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:32.345933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272d17a-1da5-4d56-bf74-d002b568394d None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:32.346482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272d17a-1da5-4d56-bf74-d002b568394d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:32.346482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272d17a-1da5-4d56-bf74-d002b568394d None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:32.348101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 941/1899] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:46:32.354329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a admin admin] The token cache is being invalidate because project fcafb472d06f4b5eb1ca07e95611b88a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:32.354707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:32.355640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-47ba4697-a5fb-4aba-86fb-c5edd94fa11a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:32.360374 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 959/1900] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:32 2022] DELETE /identity/v3/projects/fcafb472d06f4b5eb1ca07e95611b88a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:32.361620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272d17a-1da5-4d56-bf74-d002b568394d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:32.363547 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8272d17a-1da5-4d56-bf74-d002b568394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:32.374418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe1c9d-78a2-4a25-b5a3-00240b51b50d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:32.376153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe1c9d-78a2-4a25-b5a3-00240b51b50d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:32.376629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe1c9d-78a2-4a25-b5a3-00240b51b50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:32.405464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-4cfe1c9d-78a2-4a25-b5a3-00240b51b50d None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:32.407211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4cfe1c9d-78a2-4a25-b5a3-00240b51b50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:32.499576 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 960/1901] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:46:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:46:32.520760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272d17a-1da5-4d56-bf74-d002b568394d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:32.521069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe1c9d-78a2-4a25-b5a3-00240b51b50d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:32.529309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d6ce2-06d9-409b-801e-cdf3b9a093e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:32.530128 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d6ce2-06d9-409b-801e-cdf3b9a093e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:32.530672 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d6ce2-06d9-409b-801e-cdf3b9a093e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:32.531149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d6ce2-06d9-409b-801e-cdf3b9a093e4 admin admin] PATH_INFO: `/v3/users/38e2f5650c6a41e7ac723800c00ad77e` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:32.532568 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d6ce2-06d9-409b-801e-cdf3b9a093e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e2f5650c6a41e7ac723800c00ad77e)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:32.534662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d6ce2-06d9-409b-801e-cdf3b9a093e4 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:32.536844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4cfe1c9d-78a2-4a25-b5a3-00240b51b50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:32.570206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe1c9d-78a2-4a25-b5a3-00240b51b50d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:32.578199 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afe051-3648-4c35-b7d9-bf4d2c99d688 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:32.578820 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afe051-3648-4c35-b7d9-bf4d2c99d688 service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:32.579106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afe051-3648-4c35-b7d9-bf4d2c99d688 service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:32.579390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afe051-3648-4c35-b7d9-bf4d2c99d688 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:32.588894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afe051-3648-4c35-b7d9-bf4d2c99d688 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:32.590660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afe051-3648-4c35-b7d9-bf4d2c99d688 service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:32.592846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fe9d6ce2-06d9-409b-801e-cdf3b9a093e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e2f5650c6a41e7ac723800c00ad77e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:32.593511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fe9d6ce2-06d9-409b-801e-cdf3b9a093e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e2f5650c6a41e7ac723800c00ad77e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:32.603838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 961/1902] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:32.613033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fe9d6ce2-06d9-409b-801e-cdf3b9a093e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e2f5650c6a41e7ac723800c00ad77e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:32.631122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fe9d6ce2-06d9-409b-801e-cdf3b9a093e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e2f5650c6a41e7ac723800c00ad77e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:32.632766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fe9d6ce2-06d9-409b-801e-cdf3b9a093e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e2f5650c6a41e7ac723800c00ad77e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:32.653495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 942/1903] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:32 2022] DELETE /identity/v3/users/38e2f5650c6a41e7ac723800c00ad77e => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:33.060412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afe051-3648-4c35-b7d9-bf4d2c99d688 service nova] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:33.080236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afe051-3648-4c35-b7d9-bf4d2c99d688 service nova] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:33.183518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:33.206943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:33.207811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:33.208331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:33.208830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e admin admin] PATH_INFO: `/v3/projects/56b8be344aab43f3a48f7b39c5dc9a03` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:33.210275 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b8be344aab43f3a48f7b39c5dc9a03)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:33.212451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:33.262798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b8be344aab43f3a48f7b39c5dc9a03', 'request_id': 'req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:33.269849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b8be344aab43f3a48f7b39c5dc9a03', 'request_id': 'req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:33.275624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b8be344aab43f3a48f7b39c5dc9a03', 'request_id': 'req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:33.287887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e admin admin] The token cache is being invalidate because project 56b8be344aab43f3a48f7b39c5dc9a03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:33.288691 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:33.289301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e19d3fc2-551c-41e5-8181-a8b56ab2d87e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:33.290806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 962/1904] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:33 2022] DELETE /identity/v3/projects/56b8be344aab43f3a48f7b39c5dc9a03 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:33.295652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103807d9-b1b1-48b8-8110-18b141a9bea8 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:33.295941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103807d9-b1b1-48b8-8110-18b141a9bea8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:33.296221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103807d9-b1b1-48b8-8110-18b141a9bea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:33.316338 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-103807d9-b1b1-48b8-8110-18b141a9bea8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:33.317544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-103807d9-b1b1-48b8-8110-18b141a9bea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:33.375926 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 943/1905] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:33.383475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b50013-9e8f-4a22-9952-542298992cac None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:33.384316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b50013-9e8f-4a22-9952-542298992cac None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:33.384625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b50013-9e8f-4a22-9952-542298992cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:33.404556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-f0b50013-9e8f-4a22-9952-542298992cac None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:33.405822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f0b50013-9e8f-4a22-9952-542298992cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:33.414403 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 963/1906] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:33.418628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103807d9-b1b1-48b8-8110-18b141a9bea8 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:33.424630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:33.428809 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103807d9-b1b1-48b8-8110-18b141a9bea8 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:33.435211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7650e6-e758-4f60-ac8b-56aecf749fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:33.435949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7650e6-e758-4f60-ac8b-56aecf749fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:33.436233 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7650e6-e758-4f60-ac8b-56aecf749fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:33.436510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7650e6-e758-4f60-ac8b-56aecf749fd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:33.437153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7650e6-e758-4f60-ac8b-56aecf749fd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:33.437467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7650e6-e758-4f60-ac8b-56aecf749fd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:33.439086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7650e6-e758-4f60-ac8b-56aecf749fd6 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:33.450588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 944/1907] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:46:33.455016 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b50013-9e8f-4a22-9952-542298992cac None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:33.465214 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b50013-9e8f-4a22-9952-542298992cac None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:33.470649 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582c5f4-905e-4517-9610-cdf15be6cc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:33.471240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582c5f4-905e-4517-9610-cdf15be6cc86 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:33.471517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582c5f4-905e-4517-9610-cdf15be6cc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:33.471782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582c5f4-905e-4517-9610-cdf15be6cc86 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:33.472413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582c5f4-905e-4517-9610-cdf15be6cc86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:33.474346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582c5f4-905e-4517-9610-cdf15be6cc86 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:33.513575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 964/1908] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:33 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:33.514803 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7650e6-e758-4f60-ac8b-56aecf749fd6 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:33.526675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7650e6-e758-4f60-ac8b-56aecf749fd6 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:33.613559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd178b3-3e4d-440f-a6e3-5410eed46205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:33.614800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd178b3-3e4d-440f-a6e3-5410eed46205 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:33.615087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd178b3-3e4d-440f-a6e3-5410eed46205 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:33.615917 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd178b3-3e4d-440f-a6e3-5410eed46205 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:33.616920 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd178b3-3e4d-440f-a6e3-5410eed46205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:33.620069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd178b3-3e4d-440f-a6e3-5410eed46205 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:33.642482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 945/1909] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:33 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:46:33.647018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582c5f4-905e-4517-9610-cdf15be6cc86 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:33.668520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582c5f4-905e-4517-9610-cdf15be6cc86 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:33.676147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d16bb-abfa-4a73-807c-c3b0edd2a5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:33.676915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d16bb-abfa-4a73-807c-c3b0edd2a5ed admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:33.677167 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d16bb-abfa-4a73-807c-c3b0edd2a5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:33.677476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d16bb-abfa-4a73-807c-c3b0edd2a5ed admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:33.678335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d16bb-abfa-4a73-807c-c3b0edd2a5ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:33.680596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d16bb-abfa-4a73-807c-c3b0edd2a5ed admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:33.693766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 965/1910] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:33.697911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd178b3-3e4d-440f-a6e3-5410eed46205 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:33.708919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd178b3-3e4d-440f-a6e3-5410eed46205 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:33.717616 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80186f1c-8079-4957-8338-a1c312424797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:33.718204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80186f1c-8079-4957-8338-a1c312424797 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:33.718506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80186f1c-8079-4957-8338-a1c312424797 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:33.718852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80186f1c-8079-4957-8338-a1c312424797 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:33.719546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80186f1c-8079-4957-8338-a1c312424797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:33.721517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80186f1c-8079-4957-8338-a1c312424797 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:33.734352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 946/1911] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:33.738450 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d16bb-abfa-4a73-807c-c3b0edd2a5ed admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:33.752842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d16bb-abfa-4a73-807c-c3b0edd2a5ed admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:33.759466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884ae19-7eee-464f-b96c-76fcdb0bb2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:33.760169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884ae19-7eee-464f-b96c-76fcdb0bb2eb admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:33.760458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884ae19-7eee-464f-b96c-76fcdb0bb2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:33.760814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884ae19-7eee-464f-b96c-76fcdb0bb2eb admin admin] PATH_INFO: `/v3/projects/4f7664e62c514b5fbfe6717461d22a5a/users/491ead4831484c14b07d2932628cb9cc/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:33.787336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884ae19-7eee-464f-b96c-76fcdb0bb2eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f7664e62c514b5fbfe6717461d22a5a, user_id=491ead4831484c14b07d2932628cb9cc, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:33.791226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884ae19-7eee-464f-b96c-76fcdb0bb2eb admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:33.811671 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 966/1912] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:33 2022] PUT /identity/v3/projects/4f7664e62c514b5fbfe6717461d22a5a/users/491ead4831484c14b07d2932628cb9cc/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:33.823524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3e959-5342-4cca-8b05-872020224a7b None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:33.823967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3e959-5342-4cca-8b05-872020224a7b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:33.824490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3e959-5342-4cca-8b05-872020224a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:33.868358 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-2fe3e959-5342-4cca-8b05-872020224a7b None None] MFA Rules not processed for user `491ead4831484c14b07d2932628cb9cc`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:33.869574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2fe3e959-5342-4cca-8b05-872020224a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:33.956005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 947/1913] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:46:33.961522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884ae19-7eee-464f-b96c-76fcdb0bb2eb admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:34.027451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884ae19-7eee-464f-b96c-76fcdb0bb2eb admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:34.032434 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca396b-1ab7-4fab-a396-9cf3f0c17449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:34.033043 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca396b-1ab7-4fab-a396-9cf3f0c17449 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:34.033376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca396b-1ab7-4fab-a396-9cf3f0c17449 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:34.033596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca396b-1ab7-4fab-a396-9cf3f0c17449 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:34.034378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca396b-1ab7-4fab-a396-9cf3f0c17449 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:34.036334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca396b-1ab7-4fab-a396-9cf3f0c17449 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:34.064441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 967/1914] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:33 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:34.068911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3e959-5342-4cca-8b05-872020224a7b None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:34.081030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3e959-5342-4cca-8b05-872020224a7b None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:34.185178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f1620-7009-4c50-aba6-8d9780f061b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:34.185838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f1620-7009-4c50-aba6-8d9780f061b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:34.186122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f1620-7009-4c50-aba6-8d9780f061b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:34.186395 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f1620-7009-4c50-aba6-8d9780f061b7 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:34.187063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f1620-7009-4c50-aba6-8d9780f061b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:34.189363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f1620-7009-4c50-aba6-8d9780f061b7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:34.204092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:34.230327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 948/1915] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:34 2022] POST /identity/v3/users => generated 301 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:34.234560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca396b-1ab7-4fab-a396-9cf3f0c17449 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:34.249220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca396b-1ab7-4fab-a396-9cf3f0c17449 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:34.256297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985f729-8ab8-40f6-b234-763dc1b7ee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:34.257669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985f729-8ab8-40f6-b234-763dc1b7ee0d admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:34.258565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985f729-8ab8-40f6-b234-763dc1b7ee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:34.259178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985f729-8ab8-40f6-b234-763dc1b7ee0d admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:34.260048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985f729-8ab8-40f6-b234-763dc1b7ee0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:34.262794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985f729-8ab8-40f6-b234-763dc1b7ee0d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:34.264637 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:34.285349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 968/1916] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:34.291541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f1620-7009-4c50-aba6-8d9780f061b7 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:34.306243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f1620-7009-4c50-aba6-8d9780f061b7 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:34.314081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf9aea-3ab2-4fb7-b2b2-283e80a68b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:34.316733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf9aea-3ab2-4fb7-b2b2-283e80a68b0c admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:34.317323 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf9aea-3ab2-4fb7-b2b2-283e80a68b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:34.317810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf9aea-3ab2-4fb7-b2b2-283e80a68b0c admin admin] PATH_INFO: `/v3/projects/06762cb542f84d99bb71d32b7cbc71a3/users/41cef611ddc5464fb209b0a27921e5a7/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:34.353220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf9aea-3ab2-4fb7-b2b2-283e80a68b0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=06762cb542f84d99bb71d32b7cbc71a3, user_id=41cef611ddc5464fb209b0a27921e5a7, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:34.357415 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf9aea-3ab2-4fb7-b2b2-283e80a68b0c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:34.387439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 949/1917] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:34 2022] PUT /identity/v3/projects/06762cb542f84d99bb71d32b7cbc71a3/users/41cef611ddc5464fb209b0a27921e5a7/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:34.393522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985f729-8ab8-40f6-b234-763dc1b7ee0d admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:34.410934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985f729-8ab8-40f6-b234-763dc1b7ee0d admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:34.489958 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3458b3-0627-414c-8279-a56b0996181b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:34.490580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3458b3-0627-414c-8279-a56b0996181b admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:34.490858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3458b3-0627-414c-8279-a56b0996181b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:34.491128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3458b3-0627-414c-8279-a56b0996181b admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:34.491851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3458b3-0627-414c-8279-a56b0996181b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:34.493861 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3458b3-0627-414c-8279-a56b0996181b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:34.513175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 969/1918] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:34.514498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf9aea-3ab2-4fb7-b2b2-283e80a68b0c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:34.527634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf9aea-3ab2-4fb7-b2b2-283e80a68b0c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:34.532402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd10884-5f7e-4134-96f9-bbd02da02cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:34.533062 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd10884-5f7e-4134-96f9-bbd02da02cda admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:34.533346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd10884-5f7e-4134-96f9-bbd02da02cda admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:34.534535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd10884-5f7e-4134-96f9-bbd02da02cda admin admin] PATH_INFO: `/v3/projects/06762cb542f84d99bb71d32b7cbc71a3/users/41cef611ddc5464fb209b0a27921e5a7/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:34.536205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd10884-5f7e-4134-96f9-bbd02da02cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=06762cb542f84d99bb71d32b7cbc71a3, user_id=41cef611ddc5464fb209b0a27921e5a7, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:34.539751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd10884-5f7e-4134-96f9-bbd02da02cda admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:34.550938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 950/1919] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:34 2022] PUT /identity/v3/projects/06762cb542f84d99bb71d32b7cbc71a3/users/41cef611ddc5464fb209b0a27921e5a7/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:34.562421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ecc1d-d5d9-4aad-bfa3-2088d18419b7 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:34.562711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ecc1d-d5d9-4aad-bfa3-2088d18419b7 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:34.563024 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ecc1d-d5d9-4aad-bfa3-2088d18419b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:34.597375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-432ecc1d-d5d9-4aad-bfa3-2088d18419b7 None None] MFA Rules not processed for user `41cef611ddc5464fb209b0a27921e5a7`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:34.598574 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-432ecc1d-d5d9-4aad-bfa3-2088d18419b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:34.611356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd10884-5f7e-4134-96f9-bbd02da02cda admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:34.613443 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8cd10884-5f7e-4134-96f9-bbd02da02cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:34.707606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd10884-5f7e-4134-96f9-bbd02da02cda admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:34.712823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3da14-7617-4840-afcd-057bb81fcf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:34.713323 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 970/1920] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:34 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:34.714020 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3da14-7617-4840-afcd-057bb81fcf98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:34.714428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3da14-7617-4840-afcd-057bb81fcf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:34.714774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3da14-7617-4840-afcd-057bb81fcf98 admin admin] PATH_INFO: `/v3/users/25febc0733ac41edbecde156f7cb8f46` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:34.716054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3da14-7617-4840-afcd-057bb81fcf98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25febc0733ac41edbecde156f7cb8f46)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:34.718691 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3da14-7617-4840-afcd-057bb81fcf98 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:34.724696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ecc1d-d5d9-4aad-bfa3-2088d18419b7 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:34.727070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-432ecc1d-d5d9-4aad-bfa3-2088d18419b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:34.785976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-abd3da14-7617-4840-afcd-057bb81fcf98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25febc0733ac41edbecde156f7cb8f46'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:34.786661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-abd3da14-7617-4840-afcd-057bb81fcf98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25febc0733ac41edbecde156f7cb8f46'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:34.799836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-abd3da14-7617-4840-afcd-057bb81fcf98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25febc0733ac41edbecde156f7cb8f46'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:34.806036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ecc1d-d5d9-4aad-bfa3-2088d18419b7 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:34.808004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-abd3da14-7617-4840-afcd-057bb81fcf98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25febc0733ac41edbecde156f7cb8f46'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:34.808679 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-abd3da14-7617-4840-afcd-057bb81fcf98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25febc0733ac41edbecde156f7cb8f46'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:34.809756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ae4cc-1aee-445c-b11d-d08a7e609d50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:34.810428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ae4cc-1aee-445c-b11d-d08a7e609d50 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:34.810795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ae4cc-1aee-445c-b11d-d08a7e609d50 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:34.811135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ae4cc-1aee-445c-b11d-d08a7e609d50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:34.812844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-790ae4cc-1aee-445c-b11d-d08a7e609d50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:34.823855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 951/1921] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:34 2022] DELETE /identity/v3/users/25febc0733ac41edbecde156f7cb8f46 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:34.826959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ae4cc-1aee-445c-b11d-d08a7e609d50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:34.826959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ae4cc-1aee-445c-b11d-d08a7e609d50 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:34.828200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a5e9c-52d6-488c-ad1e-4c0823f41f8a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:34.828618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a5e9c-52d6-488c-ad1e-4c0823f41f8a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:34.828924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a5e9c-52d6-488c-ad1e-4c0823f41f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:34.836545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 971/1922] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:34 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:46:34.852094 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f18a5e9c-52d6-488c-ad1e-4c0823f41f8a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:34.853307 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f18a5e9c-52d6-488c-ad1e-4c0823f41f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:34.888199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:INSERT=6,keystone:SELECT=84,keystone:DELETE=10 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:34.929074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 952/1923] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:46:34.940148 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ae4cc-1aee-445c-b11d-d08a7e609d50 service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:34.942135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-790ae4cc-1aee-445c-b11d-d08a7e609d50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:34.990498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:35.021946 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a5e9c-52d6-488c-ad1e-4c0823f41f8a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:35.023974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f18a5e9c-52d6-488c-ad1e-4c0823f41f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:35.037878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a5e9c-52d6-488c-ad1e-4c0823f41f8a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:35.042805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:35.043353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:35.043649 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:35.043906 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] PATH_INFO: `/v3/users/491ead4831484c14b07d2932628cb9cc` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:35.045243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=491ead4831484c14b07d2932628cb9cc)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:35.046998 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:35.048973 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ae4cc-1aee-445c-b11d-d08a7e609d50 service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:35.098192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '491ead4831484c14b07d2932628cb9cc'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.098867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '491ead4831484c14b07d2932628cb9cc'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.107171 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080de9bb-61da-4b3e-8e5d-5241f1c8ee39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:35.107888 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080de9bb-61da-4b3e-8e5d-5241f1c8ee39 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:35.108263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080de9bb-61da-4b3e-8e5d-5241f1c8ee39 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:35.108537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080de9bb-61da-4b3e-8e5d-5241f1c8ee39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:35.111563 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '491ead4831484c14b07d2932628cb9cc'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.117821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080de9bb-61da-4b3e-8e5d-5241f1c8ee39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:35.118426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '491ead4831484c14b07d2932628cb9cc'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.119106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '491ead4831484c14b07d2932628cb9cc'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.119303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080de9bb-61da-4b3e-8e5d-5241f1c8ee39 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:35.131070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 972/1924] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:35.131957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 953/1925] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:35 2022] DELETE /identity/v3/users/491ead4831484c14b07d2932628cb9cc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:35.136482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd31db-25df-40f6-adec-a1a4ca92d826 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:35.136775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd31db-25df-40f6-adec-a1a4ca92d826 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:35.137084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd31db-25df-40f6-adec-a1a4ca92d826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:35.156866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-edfd31db-25df-40f6-adec-a1a4ca92d826 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:35.158202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-edfd31db-25df-40f6-adec-a1a4ca92d826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:35.239931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 973/1926] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:35.249744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:35.251319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:35.322972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6f3da-2ca3-480b-a7e1-0d6930f9e8d3 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:35.327876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc94e5d-29cc-4e5b-a70e-5ac59161c853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:35.328467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc94e5d-29cc-4e5b-a70e-5ac59161c853 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:35.328780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc94e5d-29cc-4e5b-a70e-5ac59161c853 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:35.328995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc94e5d-29cc-4e5b-a70e-5ac59161c853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:35.340010 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc94e5d-29cc-4e5b-a70e-5ac59161c853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:35.341534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc94e5d-29cc-4e5b-a70e-5ac59161c853 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:35.352371 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 954/1927] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:35.475168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd31db-25df-40f6-adec-a1a4ca92d826 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:35.476784 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-edfd31db-25df-40f6-adec-a1a4ca92d826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:35.509809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd31db-25df-40f6-adec-a1a4ca92d826 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:35.529091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:35.529920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:35.530281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:35.530699 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] PATH_INFO: `/v3/projects/8f6bca0f4f83430d98e51062f19d875f` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:35.532886 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6bca0f4f83430d98e51062f19d875f)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:35.537723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:35.630969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6bca0f4f83430d98e51062f19d875f', 'request_id': 'req-07c13939-98d3-4a31-8d41-6a9b93460cd5'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.637948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6bca0f4f83430d98e51062f19d875f', 'request_id': 'req-07c13939-98d3-4a31-8d41-6a9b93460cd5'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.645385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6bca0f4f83430d98e51062f19d875f', 'request_id': 'req-07c13939-98d3-4a31-8d41-6a9b93460cd5'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.667249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] The token cache is being invalidate because project 8f6bca0f4f83430d98e51062f19d875f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:35.667544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.668263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.669974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 974/1928] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:35 2022] DELETE /identity/v3/projects/8f6bca0f4f83430d98e51062f19d875f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:35.674096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc94e5d-29cc-4e5b-a70e-5ac59161c853 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:35.684510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc94e5d-29cc-4e5b-a70e-5ac59161c853 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:35.739370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:35.761956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:35.763098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:35.768351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:35.768534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] PATH_INFO: `/v3/users/66ff0624d9bc4ee98795e1d39757de8c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:35.769881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ff0624d9bc4ee98795e1d39757de8c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:35.776968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:35.838762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:35.851226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:35.863382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:35.868537 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ff0624d9bc4ee98795e1d39757de8c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.869369 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ff0624d9bc4ee98795e1d39757de8c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.872651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c13939-98d3-4a31-8d41-6a9b93460cd5 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:35.879636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:35.880304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:35.880588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:35.880894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] PATH_INFO: `/v3/projects/4f7664e62c514b5fbfe6717461d22a5a` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:35.882683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f7664e62c514b5fbfe6717461d22a5a)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:35.884677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:35.884794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ff0624d9bc4ee98795e1d39757de8c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.892014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ff0624d9bc4ee98795e1d39757de8c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.892814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ff0624d9bc4ee98795e1d39757de8c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.906076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 955/1929] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:35 2022] DELETE /identity/v3/users/66ff0624d9bc4ee98795e1d39757de8c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:35.936479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f7664e62c514b5fbfe6717461d22a5a', 'request_id': 'req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.942814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f7664e62c514b5fbfe6717461d22a5a', 'request_id': 'req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.949224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f7664e62c514b5fbfe6717461d22a5a', 'request_id': 'req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.962341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] The token cache is being invalidate because project 4f7664e62c514b5fbfe6717461d22a5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:35.962700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.963778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:35.965318 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 975/1930] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:35 2022] DELETE /identity/v3/projects/4f7664e62c514b5fbfe6717461d22a5a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:35.970025 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:35.971455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:36.043088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc6bc3-a6cb-4782-adc4-992a06c5edd4 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:36.153296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012a764-a6ae-4fc6-a66e-ca1206ecf554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:36.154239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012a764-a6ae-4fc6-a66e-ca1206ecf554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:36.154530 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012a764-a6ae-4fc6-a66e-ca1206ecf554 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:36.154921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012a764-a6ae-4fc6-a66e-ca1206ecf554 admin admin] PATH_INFO: `/v3/users/41cef611ddc5464fb209b0a27921e5a7` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:36.156099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012a764-a6ae-4fc6-a66e-ca1206ecf554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41cef611ddc5464fb209b0a27921e5a7)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:36.158261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012a764-a6ae-4fc6-a66e-ca1206ecf554 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:36.173300 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:36.216396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-a012a764-a6ae-4fc6-a66e-ca1206ecf554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41cef611ddc5464fb209b0a27921e5a7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.217161 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-a012a764-a6ae-4fc6-a66e-ca1206ecf554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41cef611ddc5464fb209b0a27921e5a7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.239063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-a012a764-a6ae-4fc6-a66e-ca1206ecf554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41cef611ddc5464fb209b0a27921e5a7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.248571 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-a012a764-a6ae-4fc6-a66e-ca1206ecf554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41cef611ddc5464fb209b0a27921e5a7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.249396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-a012a764-a6ae-4fc6-a66e-ca1206ecf554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41cef611ddc5464fb209b0a27921e5a7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.268852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 956/1931] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:35 2022] DELETE /identity/v3/users/41cef611ddc5464fb209b0a27921e5a7 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:36.277302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:36.279027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:36.293818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce449f-c4e7-48ac-91c0-24d3b529ddeb admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:36.472482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:36.473726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:36.474184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:36.474674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] PATH_INFO: `/v3/projects/67f34368087a456291d2ad4d9aa5e1ef` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:36.476603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67f34368087a456291d2ad4d9aa5e1ef)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:36.479200 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:36.544433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67f34368087a456291d2ad4d9aa5e1ef', 'request_id': 'req-a082a25d-3c0b-44bf-91cd-3d74b5acd163'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.552997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67f34368087a456291d2ad4d9aa5e1ef', 'request_id': 'req-a082a25d-3c0b-44bf-91cd-3d74b5acd163'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.561483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67f34368087a456291d2ad4d9aa5e1ef', 'request_id': 'req-a082a25d-3c0b-44bf-91cd-3d74b5acd163'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.583473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] The token cache is being invalidate because project 67f34368087a456291d2ad4d9aa5e1ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:36.583831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.584619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.588854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 976/1932] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:36 2022] DELETE /identity/v3/projects/67f34368087a456291d2ad4d9aa5e1ef => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:36.594684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecfaa6-c6c4-4786-8d83-db5a78f4c035 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:36.595389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecfaa6-c6c4-4786-8d83-db5a78f4c035 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:36.595691 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecfaa6-c6c4-4786-8d83-db5a78f4c035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:36.623313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-47ecfaa6-c6c4-4786-8d83-db5a78f4c035 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:36.624818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-47ecfaa6-c6c4-4786-8d83-db5a78f4c035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:36.672885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:36.674462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:36.701737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:36.728083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 957/1933] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:36.737955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8bf7f-fb9c-47f2-8a56-3e2f18f60b35 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:36.738590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8bf7f-fb9c-47f2-8a56-3e2f18f60b35 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:36.739820 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8bf7f-fb9c-47f2-8a56-3e2f18f60b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:36.774837 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082a25d-3c0b-44bf-91cd-3d74b5acd163 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:36.781026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-31f8bf7f-fb9c-47f2-8a56-3e2f18f60b35 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:36.783754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-31f8bf7f-fb9c-47f2-8a56-3e2f18f60b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:36.785957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e8f0b-f91f-4e1c-bdbd-ac0f8d6fc858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:36.787911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e8f0b-f91f-4e1c-bdbd-ac0f8d6fc858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:36.787911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e8f0b-f91f-4e1c-bdbd-ac0f8d6fc858 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:36.788794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e8f0b-f91f-4e1c-bdbd-ac0f8d6fc858 admin admin] PATH_INFO: `/v3/users/ed746a24940a4042b04d5fee3b1353df` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:36.792491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e8f0b-f91f-4e1c-bdbd-ac0f8d6fc858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed746a24940a4042b04d5fee3b1353df)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:36.796085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6e8f0b-f91f-4e1c-bdbd-ac0f8d6fc858 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:36.803690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 958/1934] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:36.808778 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8bf7f-fb9c-47f2-8a56-3e2f18f60b35 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:36.827291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8bf7f-fb9c-47f2-8a56-3e2f18f60b35 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:36.840663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde8d7f-6d76-4992-8c8d-1eeb683362d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:36.841295 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde8d7f-6d76-4992-8c8d-1eeb683362d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:36.841573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde8d7f-6d76-4992-8c8d-1eeb683362d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:36.841876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde8d7f-6d76-4992-8c8d-1eeb683362d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:36.842531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde8d7f-6d76-4992-8c8d-1eeb683362d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:36.845300 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde8d7f-6d76-4992-8c8d-1eeb683362d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:36.847218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde8d7f-6d76-4992-8c8d-1eeb683362d8 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:36.877644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 959/1935] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:46:36.884243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde8d7f-6d76-4992-8c8d-1eeb683362d8 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:36.909592 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde8d7f-6d76-4992-8c8d-1eeb683362d8 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:36.911575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5f6e8f0b-f91f-4e1c-bdbd-ac0f8d6fc858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed746a24940a4042b04d5fee3b1353df'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.912445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5f6e8f0b-f91f-4e1c-bdbd-ac0f8d6fc858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed746a24940a4042b04d5fee3b1353df'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.927898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5f6e8f0b-f91f-4e1c-bdbd-ac0f8d6fc858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed746a24940a4042b04d5fee3b1353df'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.928805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5f6e8f0b-f91f-4e1c-bdbd-ac0f8d6fc858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed746a24940a4042b04d5fee3b1353df'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.929435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606119b-f7a6-4c5c-a4d7-1cccb795792e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:36.930140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606119b-f7a6-4c5c-a4d7-1cccb795792e admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:36.930615 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606119b-f7a6-4c5c-a4d7-1cccb795792e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:36.930927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606119b-f7a6-4c5c-a4d7-1cccb795792e admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:36.932031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606119b-f7a6-4c5c-a4d7-1cccb795792e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:36.933999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606119b-f7a6-4c5c-a4d7-1cccb795792e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:36.949491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5f6e8f0b-f91f-4e1c-bdbd-ac0f8d6fc858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed746a24940a4042b04d5fee3b1353df'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:36.984641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 977/1936] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:36 2022] DELETE /identity/v3/users/ed746a24940a4042b04d5fee3b1353df => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:36.989591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358c668-a10b-4345-bf37-375e35567752 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:36.989955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358c668-a10b-4345-bf37-375e35567752 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:36.990265 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358c668-a10b-4345-bf37-375e35567752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.012999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 960/1937] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:36 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:37.017167 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606119b-f7a6-4c5c-a4d7-1cccb795792e admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.019513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-5358c668-a10b-4345-bf37-375e35567752 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:37.020737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-5358c668-a10b-4345-bf37-375e35567752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:37.020862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b606119b-f7a6-4c5c-a4d7-1cccb795792e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:37.077857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:37.093342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606119b-f7a6-4c5c-a4d7-1cccb795792e admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.099937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307f963-e4d9-48d6-b430-ee3fee5fa246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.101801 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307f963-e4d9-48d6-b430-ee3fee5fa246 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.102728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307f963-e4d9-48d6-b430-ee3fee5fa246 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.103338 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307f963-e4d9-48d6-b430-ee3fee5fa246 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.104224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307f963-e4d9-48d6-b430-ee3fee5fa246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.107422 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307f963-e4d9-48d6-b430-ee3fee5fa246 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.124287 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 978/1938] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:37.126398 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358c668-a10b-4345-bf37-375e35567752 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.148180 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-5358c668-a10b-4345-bf37-375e35567752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:37.171488 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 961/1939] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:37 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:46:37.172514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307f963-e4d9-48d6-b430-ee3fee5fa246 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.174939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0307f963-e4d9-48d6-b430-ee3fee5fa246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:37.176976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358c668-a10b-4345-bf37-375e35567752 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.182494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.182731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.182942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.183211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] PATH_INFO: `/v3/projects/06762cb542f84d99bb71d32b7cbc71a3` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.184686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=06762cb542f84d99bb71d32b7cbc71a3)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.186666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.229014 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:37.246444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06762cb542f84d99bb71d32b7cbc71a3', 'request_id': 'req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:37.252844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06762cb542f84d99bb71d32b7cbc71a3', 'request_id': 'req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:37.253214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307f963-e4d9-48d6-b430-ee3fee5fa246 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.260265 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06762cb542f84d99bb71d32b7cbc71a3', 'request_id': 'req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:37.275955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] The token cache is being invalidate because project 06762cb542f84d99bb71d32b7cbc71a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:37.276341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:37.277000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:37.282286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 979/1940] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:37 2022] DELETE /identity/v3/projects/06762cb542f84d99bb71d32b7cbc71a3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:37.282286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.282286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:37.315237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1b55e-8635-4504-9fbf-5dbd7b2b38e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.315951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1b55e-8635-4504-9fbf-5dbd7b2b38e5 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.316240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1b55e-8635-4504-9fbf-5dbd7b2b38e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.316603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1b55e-8635-4504-9fbf-5dbd7b2b38e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.318902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4db1b55e-8635-4504-9fbf-5dbd7b2b38e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:37.354852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1821e9b-fde5-4d4b-a2b1-0619f0547afa admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.362581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e80d6b-dee3-4868-bf56-702a43145a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.363507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e80d6b-dee3-4868-bf56-702a43145a33 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.363790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e80d6b-dee3-4868-bf56-702a43145a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.364350 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e80d6b-dee3-4868-bf56-702a43145a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.365002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e80d6b-dee3-4868-bf56-702a43145a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.367196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e80d6b-dee3-4868-bf56-702a43145a33 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.378254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 980/1941] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:37.379305 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1bf45-45f5-4ffa-bd7a-ad05929030b8 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.379591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1bf45-45f5-4ffa-bd7a-ad05929030b8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.379865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1bf45-45f5-4ffa-bd7a-ad05929030b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.387523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1b55e-8635-4504-9fbf-5dbd7b2b38e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.389955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1b55e-8635-4504-9fbf-5dbd7b2b38e5 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.408122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 962/1942] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:37.409043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1b55e-8635-4504-9fbf-5dbd7b2b38e5 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.418894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1b55e-8635-4504-9fbf-5dbd7b2b38e5 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.423889 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc54ed-46ed-4beb-b9b9-1e2ee7823a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.424526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc54ed-46ed-4beb-b9b9-1e2ee7823a16 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.424806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc54ed-46ed-4beb-b9b9-1e2ee7823a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.425077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc54ed-46ed-4beb-b9b9-1e2ee7823a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.425737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc54ed-46ed-4beb-b9b9-1e2ee7823a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.425971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-dda1bf45-45f5-4ffa-bd7a-ad05929030b8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:37.427336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-dda1bf45-45f5-4ffa-bd7a-ad05929030b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:37.427449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc54ed-46ed-4beb-b9b9-1e2ee7823a16 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.439783 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 963/1943] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:37.440004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 981/1944] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:37.443533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc54ed-46ed-4beb-b9b9-1e2ee7823a16 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.448677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94891402-30e7-4c5e-8708-df72d9dfb835 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.448968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94891402-30e7-4c5e-8708-df72d9dfb835 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.449244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94891402-30e7-4c5e-8708-df72d9dfb835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.457413 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc54ed-46ed-4beb-b9b9-1e2ee7823a16 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.463150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773c8e0-4930-4d94-b44d-5200a28b4124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.463868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773c8e0-4930-4d94-b44d-5200a28b4124 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.464284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773c8e0-4930-4d94-b44d-5200a28b4124 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.464564 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773c8e0-4930-4d94-b44d-5200a28b4124 admin admin] PATH_INFO: `/v3/projects/8d12a89b2af84e1c9b9512f11debbc40/users/52d0a60babe54c9283ea3bb400e48100/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.470185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-94891402-30e7-4c5e-8708-df72d9dfb835 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:37.471562 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-94891402-30e7-4c5e-8708-df72d9dfb835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:37.481987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 982/1945] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:37.485654 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94891402-30e7-4c5e-8708-df72d9dfb835 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.497292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94891402-30e7-4c5e-8708-df72d9dfb835 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.500930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773c8e0-4930-4d94-b44d-5200a28b4124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d12a89b2af84e1c9b9512f11debbc40, user_id=52d0a60babe54c9283ea3bb400e48100, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.503316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbed1d-601f-47ee-9d9e-82c31d6a1bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.503960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbed1d-601f-47ee-9d9e-82c31d6a1bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.504295 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbed1d-601f-47ee-9d9e-82c31d6a1bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.504853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbed1d-601f-47ee-9d9e-82c31d6a1bb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.505556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbed1d-601f-47ee-9d9e-82c31d6a1bb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:37.505888 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbed1d-601f-47ee-9d9e-82c31d6a1bb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.507651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773c8e0-4930-4d94-b44d-5200a28b4124 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.508631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbed1d-601f-47ee-9d9e-82c31d6a1bb4 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.533778 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 964/1946] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:37 2022] PUT /identity/v3/projects/8d12a89b2af84e1c9b9512f11debbc40/users/52d0a60babe54c9283ea3bb400e48100/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:37.535108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 983/1947] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:46:37.537084 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773c8e0-4930-4d94-b44d-5200a28b4124 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.539355 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000bb03-d96f-4a24-823b-f77c82a24ad7 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.539728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000bb03-d96f-4a24-823b-f77c82a24ad7 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.540182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000bb03-d96f-4a24-823b-f77c82a24ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.553419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:37.570617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-5000bb03-d96f-4a24-823b-f77c82a24ad7 None None] MFA Rules not processed for user `52d0a60babe54c9283ea3bb400e48100`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:37.572201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-5000bb03-d96f-4a24-823b-f77c82a24ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:37.634232 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773c8e0-4930-4d94-b44d-5200a28b4124 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.641124 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8348d-0203-4ecc-8d29-8b63995e1f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.641902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8348d-0203-4ecc-8d29-8b63995e1f09 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.642178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8348d-0203-4ecc-8d29-8b63995e1f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.642878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8348d-0203-4ecc-8d29-8b63995e1f09 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.643510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8348d-0203-4ecc-8d29-8b63995e1f09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.645700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8348d-0203-4ecc-8d29-8b63995e1f09 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.653252 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 984/1948] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:37.657535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000bb03-d96f-4a24-823b-f77c82a24ad7 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.668883 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000bb03-d96f-4a24-823b-f77c82a24ad7 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.673735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d28b8-5dfd-4743-87d5-f7e758a11db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.674344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d28b8-5dfd-4743-87d5-f7e758a11db2 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.674623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d28b8-5dfd-4743-87d5-f7e758a11db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.674893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d28b8-5dfd-4743-87d5-f7e758a11db2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.675526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d28b8-5dfd-4743-87d5-f7e758a11db2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.677790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d28b8-5dfd-4743-87d5-f7e758a11db2 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.691483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 965/1949] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:37 2022] POST /identity/v3/projects => generated 367 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:37.692570 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8348d-0203-4ecc-8d29-8b63995e1f09 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.696382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:37.724299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 985/1950] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:37 2022] POST /identity/v3/projects => generated 365 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:46:37.725366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d28b8-5dfd-4743-87d5-f7e758a11db2 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.780000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8348d-0203-4ecc-8d29-8b63995e1f09 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.818494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d28b8-5dfd-4743-87d5-f7e758a11db2 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.824387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7bd13-a548-4fa7-9d26-ce657f774c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.825182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7bd13-a548-4fa7-9d26-ce657f774c13 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.825604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7bd13-a548-4fa7-9d26-ce657f774c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.826129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7bd13-a548-4fa7-9d26-ce657f774c13 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.826907 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7bd13-a548-4fa7-9d26-ce657f774c13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.829384 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c7bd13-a548-4fa7-9d26-ce657f774c13 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.853971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 986/1951] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:37 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:37.854734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7bd13-a548-4fa7-9d26-ce657f774c13 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.856438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f1c7bd13-a548-4fa7-9d26-ce657f774c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:37.859458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8ef8-4bac-4eee-a234-8cb7a04df8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.860256 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8ef8-4bac-4eee-a234-8cb7a04df8b2 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.860542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8ef8-4bac-4eee-a234-8cb7a04df8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.860821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8ef8-4bac-4eee-a234-8cb7a04df8b2 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.861470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd8ef8-4bac-4eee-a234-8cb7a04df8b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.863506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd8ef8-4bac-4eee-a234-8cb7a04df8b2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.870790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c7bd13-a548-4fa7-9d26-ce657f774c13 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.876779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.877534 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.877977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.878286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] PATH_INFO: `/v3/projects/547e5b4cc87d434fb938eb6409ea1f34` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.879568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=547e5b4cc87d434fb938eb6409ea1f34)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.881726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.902102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 966/1952] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:37 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:37.903133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8ef8-4bac-4eee-a234-8cb7a04df8b2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.920331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8ef8-4bac-4eee-a234-8cb7a04df8b2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.930294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18ac35-0f07-4b08-9b06-08206f086c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:37.931614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18ac35-0f07-4b08-9b06-08206f086c8c admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:37.932053 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18ac35-0f07-4b08-9b06-08206f086c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:37.932304 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18ac35-0f07-4b08-9b06-08206f086c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:37.932964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18ac35-0f07-4b08-9b06-08206f086c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:37.934671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18ac35-0f07-4b08-9b06-08206f086c8c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:37.949734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 967/1953] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:37.949734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18ac35-0f07-4b08-9b06-08206f086c8c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.950172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '547e5b4cc87d434fb938eb6409ea1f34', 'request_id': 'req-6e0baf77-00d7-426c-886c-d40ad10a6357'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:37.958518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '547e5b4cc87d434fb938eb6409ea1f34', 'request_id': 'req-6e0baf77-00d7-426c-886c-d40ad10a6357'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:37.960119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18ac35-0f07-4b08-9b06-08206f086c8c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:37.968903 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '547e5b4cc87d434fb938eb6409ea1f34', 'request_id': 'req-6e0baf77-00d7-426c-886c-d40ad10a6357'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:37.988320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:37.988614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] The token cache is being invalidate because project 547e5b4cc87d434fb938eb6409ea1f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:37.989735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:37.990129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:37.993572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 987/1954] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:37 2022] DELETE /identity/v3/projects/547e5b4cc87d434fb938eb6409ea1f34 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:37.994314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:37.997565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:38.059019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cd791-d310-4760-8f00-9711c3241e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:38.059791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cd791-d310-4760-8f00-9711c3241e63 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:38.060122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cd791-d310-4760-8f00-9711c3241e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:38.060417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cd791-d310-4760-8f00-9711c3241e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:38.061078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5cd791-d310-4760-8f00-9711c3241e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:38.062770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5cd791-d310-4760-8f00-9711c3241e63 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:38.075628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 968/1955] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:38.077029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cd791-d310-4760-8f00-9711c3241e63 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:38.079531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8a5cd791-d310-4760-8f00-9711c3241e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:38.095542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cd791-d310-4760-8f00-9711c3241e63 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:38.101224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:38.102194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:38.102563 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:38.103170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] PATH_INFO: `/v3/users/bc89839ffc8e47d19d19b36de72fd290` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:38.104222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc89839ffc8e47d19d19b36de72fd290)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:38.106368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:38.152597 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0baf77-00d7-426c-886c-d40ad10a6357 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:38.163754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933a1be-24dd-4816-903f-d3f589c828e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:38.164452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933a1be-24dd-4816-903f-d3f589c828e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:38.164741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933a1be-24dd-4816-903f-d3f589c828e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:38.165022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933a1be-24dd-4816-903f-d3f589c828e6 admin admin] PATH_INFO: `/v3/projects/c2e388f12e7b47389145b43e205f191a/users/5cf7ea8ebc1649888006c6ebff136791/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:38.210290 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc89839ffc8e47d19d19b36de72fd290'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:38.211215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc89839ffc8e47d19d19b36de72fd290'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:38.225918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:38.227662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc89839ffc8e47d19d19b36de72fd290'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:38.237318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc89839ffc8e47d19d19b36de72fd290'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:38.238058 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933a1be-24dd-4816-903f-d3f589c828e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e388f12e7b47389145b43e205f191a, user_id=5cf7ea8ebc1649888006c6ebff136791, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:38.238610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc89839ffc8e47d19d19b36de72fd290'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:38.244687 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933a1be-24dd-4816-903f-d3f589c828e6 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:38.263160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 969/1956] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:38 2022] DELETE /identity/v3/users/bc89839ffc8e47d19d19b36de72fd290 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:38.263160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:38.265904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:38.303327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 988/1957] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:37 2022] PUT /identity/v3/projects/c2e388f12e7b47389145b43e205f191a/users/5cf7ea8ebc1649888006c6ebff136791/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:38.304507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933a1be-24dd-4816-903f-d3f589c828e6 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:38.311471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3933a1be-24dd-4816-903f-d3f589c828e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:38.408380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6b223-3e1f-4245-8afa-819067087ee3 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:38.464223 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:38.489480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933a1be-24dd-4816-903f-d3f589c828e6 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:38.497840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a925be3-15da-4266-861a-6f676ae09ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:38.499595 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a925be3-15da-4266-861a-6f676ae09ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:38.499905 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a925be3-15da-4266-861a-6f676ae09ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:38.500183 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a925be3-15da-4266-861a-6f676ae09ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:38.500943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb606d5-45b8-4ddd-93be-61ff93320586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:38.501190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a925be3-15da-4266-861a-6f676ae09ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:38.501723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb606d5-45b8-4ddd-93be-61ff93320586 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:38.502060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb606d5-45b8-4ddd-93be-61ff93320586 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:38.502362 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb606d5-45b8-4ddd-93be-61ff93320586 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:38.503168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb606d5-45b8-4ddd-93be-61ff93320586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:38.503272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a925be3-15da-4266-861a-6f676ae09ed8 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:38.504786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb606d5-45b8-4ddd-93be-61ff93320586 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:38.517234 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 970/1958] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:38.522357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a925be3-15da-4266-861a-6f676ae09ed8 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:38.532321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 989/1959] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:38.533260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb606d5-45b8-4ddd-93be-61ff93320586 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:38.544188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb606d5-45b8-4ddd-93be-61ff93320586 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:38.550556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a925be3-15da-4266-861a-6f676ae09ed8 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:38.578903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f518de9-e0ef-4b20-914b-571697007fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:38.579776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f518de9-e0ef-4b20-914b-571697007fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:38.580060 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f518de9-e0ef-4b20-914b-571697007fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:38.580338 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f518de9-e0ef-4b20-914b-571697007fd3 admin admin] PATH_INFO: `/v3/projects/f8169a00e0714eceb0d7337f691254f5/users/d08e5ace26c44f6b88f34cb213db28a4/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:38.591301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870dcc0-a33e-4120-9382-9a5361a6f61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:38.592057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870dcc0-a33e-4120-9382-9a5361a6f61e admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:38.592403 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870dcc0-a33e-4120-9382-9a5361a6f61e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:38.592742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870dcc0-a33e-4120-9382-9a5361a6f61e admin admin] PATH_INFO: `/v3/projects/c2e388f12e7b47389145b43e205f191a/users/5cf7ea8ebc1649888006c6ebff136791/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:38.600914 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870dcc0-a33e-4120-9382-9a5361a6f61e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e388f12e7b47389145b43e205f191a, user_id=5cf7ea8ebc1649888006c6ebff136791, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:38.604200 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870dcc0-a33e-4120-9382-9a5361a6f61e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:38.631150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 990/1960] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:38 2022] PUT /identity/v3/projects/c2e388f12e7b47389145b43e205f191a/users/5cf7ea8ebc1649888006c6ebff136791/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:38.636653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f518de9-e0ef-4b20-914b-571697007fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8169a00e0714eceb0d7337f691254f5, user_id=d08e5ace26c44f6b88f34cb213db28a4, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:38.640795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f518de9-e0ef-4b20-914b-571697007fd3 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:38.642773 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2510a-107d-4ee8-b53f-a0bdce517666 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:38.643135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2510a-107d-4ee8-b53f-a0bdce517666 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:38.643471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2510a-107d-4ee8-b53f-a0bdce517666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:38.672156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 971/1961] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:38 2022] PUT /identity/v3/projects/f8169a00e0714eceb0d7337f691254f5/users/d08e5ace26c44f6b88f34cb213db28a4/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:38.684134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd09ee9-7a4c-4ef1-9655-5121f2ba458d None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:38.684613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd09ee9-7a4c-4ef1-9655-5121f2ba458d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:38.685069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd09ee9-7a4c-4ef1-9655-5121f2ba458d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:38.703122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-ced2510a-107d-4ee8-b53f-a0bdce517666 None None] MFA Rules not processed for user `5cf7ea8ebc1649888006c6ebff136791`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:38.704441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ced2510a-107d-4ee8-b53f-a0bdce517666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:38.748047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-abd09ee9-7a4c-4ef1-9655-5121f2ba458d None None] MFA Rules not processed for user `d08e5ace26c44f6b88f34cb213db28a4`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:38.751861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-abd09ee9-7a4c-4ef1-9655-5121f2ba458d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:38.840230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 991/1962] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:38.841548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2510a-107d-4ee8-b53f-a0bdce517666 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:38.843805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ced2510a-107d-4ee8-b53f-a0bdce517666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:38.922368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 972/1963] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:46:38.927783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2510a-107d-4ee8-b53f-a0bdce517666 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:38.935260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd09ee9-7a4c-4ef1-9655-5121f2ba458d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:38.938112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efc118-e349-4d2b-9c9a-f99af9c4b126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:38.939480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efc118-e349-4d2b-9c9a-f99af9c4b126 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:38.940761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efc118-e349-4d2b-9c9a-f99af9c4b126 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:38.941129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efc118-e349-4d2b-9c9a-f99af9c4b126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:38.967603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efc118-e349-4d2b-9c9a-f99af9c4b126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:38.971171 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5efc118-e349-4d2b-9c9a-f99af9c4b126 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:39.009930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 992/1964] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:39.057472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd09ee9-7a4c-4ef1-9655-5121f2ba458d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:39.058856 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efc118-e349-4d2b-9c9a-f99af9c4b126 service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:39.060427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c5efc118-e349-4d2b-9c9a-f99af9c4b126 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:39.065673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec067838-9877-477e-a82e-529cabae6edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:39.066690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec067838-9877-477e-a82e-529cabae6edf admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.067222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec067838-9877-477e-a82e-529cabae6edf admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.067329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:39.068913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec067838-9877-477e-a82e-529cabae6edf admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:39.071814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec067838-9877-477e-a82e-529cabae6edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:39.075200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec067838-9877-477e-a82e-529cabae6edf admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:39.082416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efc118-e349-4d2b-9c9a-f99af9c4b126 service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:39.090310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9e210-7f7d-4f34-a4e0-77f6e39e635e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:39.091802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9e210-7f7d-4f34-a4e0-77f6e39e635e service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.091944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9e210-7f7d-4f34-a4e0-77f6e39e635e service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.092257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9e210-7f7d-4f34-a4e0-77f6e39e635e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:39.095631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a6a9e210-7f7d-4f34-a4e0-77f6e39e635e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:39.115558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 973/1965] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:38 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:39.115558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec067838-9877-477e-a82e-529cabae6edf admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:39.115558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ec067838-9877-477e-a82e-529cabae6edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:39.149379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:39.184811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9e210-7f7d-4f34-a4e0-77f6e39e635e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:39.186486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9e210-7f7d-4f34-a4e0-77f6e39e635e service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:39.195905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 993/1966] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:39.196705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9e210-7f7d-4f34-a4e0-77f6e39e635e service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:39.197970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec067838-9877-477e-a82e-529cabae6edf admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:39.205734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:39.205734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.205734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.205734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] PATH_INFO: `/v3/projects/0c9336a58cdc4ab795ee01190c9fdc42` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:39.206335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9336a58cdc4ab795ee01190c9fdc42)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:39.207161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9e210-7f7d-4f34-a4e0-77f6e39e635e service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:39.208259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:39.219172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b8a0d-6620-4899-85e4-087ac3e36dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:39.220543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b8a0d-6620-4899-85e4-087ac3e36dbc admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.220832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b8a0d-6620-4899-85e4-087ac3e36dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.221590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b8a0d-6620-4899-85e4-087ac3e36dbc admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:39.222800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b8a0d-6620-4899-85e4-087ac3e36dbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:39.226207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b8a0d-6620-4899-85e4-087ac3e36dbc admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:39.257184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 994/1967] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:39 2022] POST /identity/v3/users => generated 290 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:39.261424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b8a0d-6620-4899-85e4-087ac3e36dbc admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:39.275035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b8a0d-6620-4899-85e4-087ac3e36dbc admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:39.319176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9336a58cdc4ab795ee01190c9fdc42', 'request_id': 'req-5283a5a4-da76-4254-b8ce-3db187013c72'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:39.330930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9336a58cdc4ab795ee01190c9fdc42', 'request_id': 'req-5283a5a4-da76-4254-b8ce-3db187013c72'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:39.338407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9336a58cdc4ab795ee01190c9fdc42', 'request_id': 'req-5283a5a4-da76-4254-b8ce-3db187013c72'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:39.358075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] The token cache is being invalidate because project 0c9336a58cdc4ab795ee01190c9fdc42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:39.358481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:39.359229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:39.361769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 974/1968] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:39 2022] DELETE /identity/v3/projects/0c9336a58cdc4ab795ee01190c9fdc42 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:39.381079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e1fb6-6e8c-47a5-b8f0-063ad53a467e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:39.381918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e1fb6-6e8c-47a5-b8f0-063ad53a467e admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.382202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e1fb6-6e8c-47a5-b8f0-063ad53a467e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.382640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e1fb6-6e8c-47a5-b8f0-063ad53a467e admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:39.383323 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e1fb6-6e8c-47a5-b8f0-063ad53a467e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:39.385202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e1fb6-6e8c-47a5-b8f0-063ad53a467e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:39.413782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 995/1969] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:39.416500 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:39.418538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:39.449155 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283a5a4-da76-4254-b8ce-3db187013c72 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:39.461399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f7ae3-0655-4bdd-ab8f-a0f9fb6d293a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:39.461971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f7ae3-0655-4bdd-ab8f-a0f9fb6d293a admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.462461 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f7ae3-0655-4bdd-ab8f-a0f9fb6d293a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.462830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f7ae3-0655-4bdd-ab8f-a0f9fb6d293a admin admin] PATH_INFO: `/v3/projects/370c3ffafe3e44e0afa195511b2e63f5/users/a492e628fc7a47d7a4fb4f603995f55f/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:39.491968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f7ae3-0655-4bdd-ab8f-a0f9fb6d293a admin admin] RBAC: Authorizing `identity:create_grant(project_id=370c3ffafe3e44e0afa195511b2e63f5, user_id=a492e628fc7a47d7a4fb4f603995f55f, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:39.495006 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f7ae3-0655-4bdd-ab8f-a0f9fb6d293a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:39.521956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 975/1970] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:39 2022] PUT /identity/v3/projects/370c3ffafe3e44e0afa195511b2e63f5/users/a492e628fc7a47d7a4fb4f603995f55f/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:39.526776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e1fb6-6e8c-47a5-b8f0-063ad53a467e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:39.541811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e1fb6-6e8c-47a5-b8f0-063ad53a467e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:39.712361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8eb13-6c57-47aa-aad1-d10c0ae3c8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:39.712513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8eb13-6c57-47aa-aad1-d10c0ae3c8cb admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.712743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8eb13-6c57-47aa-aad1-d10c0ae3c8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.713017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8eb13-6c57-47aa-aad1-d10c0ae3c8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:39.713667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8eb13-6c57-47aa-aad1-d10c0ae3c8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:39.715704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8eb13-6c57-47aa-aad1-d10c0ae3c8cb admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:39.729366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 996/1971] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:39.730541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f7ae3-0655-4bdd-ab8f-a0f9fb6d293a admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:39.744432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f7ae3-0655-4bdd-ab8f-a0f9fb6d293a admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:39.755677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d7588-7ddc-4513-977d-642dd85b1d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:39.756196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d7588-7ddc-4513-977d-642dd85b1d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.756486 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d7588-7ddc-4513-977d-642dd85b1d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.756830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d7588-7ddc-4513-977d-642dd85b1d6e admin admin] PATH_INFO: `/v3/projects/370c3ffafe3e44e0afa195511b2e63f5/users/a492e628fc7a47d7a4fb4f603995f55f/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:39.760168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d7588-7ddc-4513-977d-642dd85b1d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=370c3ffafe3e44e0afa195511b2e63f5, user_id=a492e628fc7a47d7a4fb4f603995f55f, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:39.762942 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d7588-7ddc-4513-977d-642dd85b1d6e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:39.774945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 976/1972] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:39 2022] PUT /identity/v3/projects/370c3ffafe3e44e0afa195511b2e63f5/users/a492e628fc7a47d7a4fb4f603995f55f/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:39.787388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31719b9e-c8fa-4a82-8cde-1c47acccb09b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.787756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31719b9e-c8fa-4a82-8cde-1c47acccb09b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.788061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31719b9e-c8fa-4a82-8cde-1c47acccb09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:39.825647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-31719b9e-c8fa-4a82-8cde-1c47acccb09b None None] MFA Rules not processed for user `a492e628fc7a47d7a4fb4f603995f55f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:39.827026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-31719b9e-c8fa-4a82-8cde-1c47acccb09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:39.831508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:39.913444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 997/1973] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:39.925055 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d7588-7ddc-4513-977d-642dd85b1d6e admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:39.926884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6a3d7588-7ddc-4513-977d-642dd85b1d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:39.985643 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d7588-7ddc-4513-977d-642dd85b1d6e admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:39.989310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d92f0-83e9-4afa-8c66-ee9a1868e2fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:39.989954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d92f0-83e9-4afa-8c66-ee9a1868e2fe service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:39.990239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d92f0-83e9-4afa-8c66-ee9a1868e2fe service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:39.990516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d92f0-83e9-4afa-8c66-ee9a1868e2fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:40.000642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d92f0-83e9-4afa-8c66-ee9a1868e2fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:40.002133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d92f0-83e9-4afa-8c66-ee9a1868e2fe service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:40.011724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 977/1974] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:40.108726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31719b9e-c8fa-4a82-8cde-1c47acccb09b None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:40.118794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31719b9e-c8fa-4a82-8cde-1c47acccb09b None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:40.122653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de24c0-bdcc-4153-80f3-66387af66747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:40.123261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de24c0-bdcc-4153-80f3-66387af66747 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:40.123539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de24c0-bdcc-4153-80f3-66387af66747 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:40.123809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de24c0-bdcc-4153-80f3-66387af66747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:40.125616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-04de24c0-bdcc-4153-80f3-66387af66747 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:40.186890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de24c0-bdcc-4153-80f3-66387af66747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:40.188267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de24c0-bdcc-4153-80f3-66387af66747 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:40.197396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 998/1975] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:42.926871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d92f0-83e9-4afa-8c66-ee9a1868e2fe service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:42.928612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a00d92f0-83e9-4afa-8c66-ee9a1868e2fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:42.956738 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:43.030973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d92f0-83e9-4afa-8c66-ee9a1868e2fe service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:43.039392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:43.040044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:43.040319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:43.040588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] PATH_INFO: `/v3/users/52d0a60babe54c9283ea3bb400e48100` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:43.042073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d0a60babe54c9283ea3bb400e48100)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:43.044040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:43.095600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d0a60babe54c9283ea3bb400e48100'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.096496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d0a60babe54c9283ea3bb400e48100'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.111550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d0a60babe54c9283ea3bb400e48100'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.118619 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d0a60babe54c9283ea3bb400e48100'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.119450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d0a60babe54c9283ea3bb400e48100'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.137134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 978/1976] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:42 2022] DELETE /identity/v3/users/52d0a60babe54c9283ea3bb400e48100 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:43.141652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59400506-2372-4f53-acfe-9af55abd9fce None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:43.142260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59400506-2372-4f53-acfe-9af55abd9fce None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:43.142260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59400506-2372-4f53-acfe-9af55abd9fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:43.162955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-59400506-2372-4f53-acfe-9af55abd9fce None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:43.164160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-59400506-2372-4f53-acfe-9af55abd9fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:43.222665 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 999/1977] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:46:43.232153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:43.233721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:43.298418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0aa5d-5bda-4a0c-8860-6cc56ae46681 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:43.302056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e0737-3e13-490e-b7de-5b9b038887cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:43.302661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e0737-3e13-490e-b7de-5b9b038887cd service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:43.302943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e0737-3e13-490e-b7de-5b9b038887cd service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:43.303214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e0737-3e13-490e-b7de-5b9b038887cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:43.330320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e0737-3e13-490e-b7de-5b9b038887cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:43.331776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e0737-3e13-490e-b7de-5b9b038887cd service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:43.341441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 979/1978] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:43.666873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59400506-2372-4f53-acfe-9af55abd9fce None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:43.671731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-59400506-2372-4f53-acfe-9af55abd9fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:43.688104 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59400506-2372-4f53-acfe-9af55abd9fce None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:43.693563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:43.698489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:43.698734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:43.699102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] PATH_INFO: `/v3/projects/8d12a89b2af84e1c9b9512f11debbc40` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:43.700993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d12a89b2af84e1c9b9512f11debbc40)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:43.703198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:43.761580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:43.782945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d12a89b2af84e1c9b9512f11debbc40', 'request_id': 'req-a7262013-c01e-4772-95f1-e684a5e3a90c'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.789269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d12a89b2af84e1c9b9512f11debbc40', 'request_id': 'req-a7262013-c01e-4772-95f1-e684a5e3a90c'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.794845 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d12a89b2af84e1c9b9512f11debbc40', 'request_id': 'req-a7262013-c01e-4772-95f1-e684a5e3a90c'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.808290 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] The token cache is being invalidate because project 8d12a89b2af84e1c9b9512f11debbc40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:43.808586 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.809240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.810907 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1000/1979] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:43 2022] DELETE /identity/v3/projects/8d12a89b2af84e1c9b9512f11debbc40 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:43.815087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e0737-3e13-490e-b7de-5b9b038887cd service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:43.825347 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e0737-3e13-490e-b7de-5b9b038887cd service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:43.866472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:43.885346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bc2a2-e675-41b9-bbe6-c137aa323c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:43.886014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bc2a2-e675-41b9-bbe6-c137aa323c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:43.886299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bc2a2-e675-41b9-bbe6-c137aa323c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:43.886576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bc2a2-e675-41b9-bbe6-c137aa323c4f admin admin] PATH_INFO: `/v3/users/5cf7ea8ebc1649888006c6ebff136791` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:43.887689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bc2a2-e675-41b9-bbe6-c137aa323c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cf7ea8ebc1649888006c6ebff136791)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:43.889787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bc2a2-e675-41b9-bbe6-c137aa323c4f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:43.934818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f47bc2a2-e675-41b9-bbe6-c137aa323c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cf7ea8ebc1649888006c6ebff136791'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.935398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f47bc2a2-e675-41b9-bbe6-c137aa323c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cf7ea8ebc1649888006c6ebff136791'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.948088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f47bc2a2-e675-41b9-bbe6-c137aa323c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cf7ea8ebc1649888006c6ebff136791'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.955937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f47bc2a2-e675-41b9-bbe6-c137aa323c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cf7ea8ebc1649888006c6ebff136791'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.956700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-f47bc2a2-e675-41b9-bbe6-c137aa323c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cf7ea8ebc1649888006c6ebff136791'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:43.972205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 980/1980] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:43 2022] DELETE /identity/v3/users/5cf7ea8ebc1649888006c6ebff136791 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:44.327004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:44.329042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:44.342804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7262013-c01e-4772-95f1-e684a5e3a90c admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:44.407692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692ed60-684c-4e0a-9514-61053d36ce55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:44.408328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692ed60-684c-4e0a-9514-61053d36ce55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:44.408611 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692ed60-684c-4e0a-9514-61053d36ce55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:44.408883 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692ed60-684c-4e0a-9514-61053d36ce55 admin admin] PATH_INFO: `/v3/projects/c2e388f12e7b47389145b43e205f191a` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:44.410043 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692ed60-684c-4e0a-9514-61053d36ce55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e388f12e7b47389145b43e205f191a)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:44.411951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692ed60-684c-4e0a-9514-61053d36ce55 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:44.465898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9692ed60-684c-4e0a-9514-61053d36ce55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e388f12e7b47389145b43e205f191a', 'request_id': 'req-9692ed60-684c-4e0a-9514-61053d36ce55'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:44.472957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9692ed60-684c-4e0a-9514-61053d36ce55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e388f12e7b47389145b43e205f191a', 'request_id': 'req-9692ed60-684c-4e0a-9514-61053d36ce55'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:44.481082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9692ed60-684c-4e0a-9514-61053d36ce55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e388f12e7b47389145b43e205f191a', 'request_id': 'req-9692ed60-684c-4e0a-9514-61053d36ce55'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:44.501144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9692ed60-684c-4e0a-9514-61053d36ce55 admin admin] The token cache is being invalidate because project c2e388f12e7b47389145b43e205f191a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:44.501502 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9692ed60-684c-4e0a-9514-61053d36ce55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:44.502222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9692ed60-684c-4e0a-9514-61053d36ce55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:44.503986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1001/1981] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:44 2022] DELETE /identity/v3/projects/c2e388f12e7b47389145b43e205f191a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:44.508958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf71096-dfac-4006-966a-aaf96c3b3022 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:44.509514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf71096-dfac-4006-966a-aaf96c3b3022 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:44.509937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf71096-dfac-4006-966a-aaf96c3b3022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:44.532064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-abf71096-dfac-4006-966a-aaf96c3b3022 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:44.533463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-abf71096-dfac-4006-966a-aaf96c3b3022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:44.613960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 981/1982] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:44.625837 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a9206-6b11-4a22-a975-9fd72331084c None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:44.626126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a9206-6b11-4a22-a975-9fd72331084c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:44.626405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a9206-6b11-4a22-a975-9fd72331084c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:44.648220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-560a9206-6b11-4a22-a975-9fd72331084c None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:44.649437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-560a9206-6b11-4a22-a975-9fd72331084c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:44.657482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1002/1983] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:44.661930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf71096-dfac-4006-966a-aaf96c3b3022 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:44.673528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf71096-dfac-4006-966a-aaf96c3b3022 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:44.679898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae73aa1-7289-4fc4-a400-c64fc03a5901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:44.680539 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae73aa1-7289-4fc4-a400-c64fc03a5901 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:44.680844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae73aa1-7289-4fc4-a400-c64fc03a5901 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:44.681095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae73aa1-7289-4fc4-a400-c64fc03a5901 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:44.682619 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae73aa1-7289-4fc4-a400-c64fc03a5901 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:44.682619 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae73aa1-7289-4fc4-a400-c64fc03a5901 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:44.683841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae73aa1-7289-4fc4-a400-c64fc03a5901 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:44.726328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 982/1984] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:46:44.727934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a9206-6b11-4a22-a975-9fd72331084c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:44.763011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a9206-6b11-4a22-a975-9fd72331084c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:44.778753 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0340f67-04cb-48dd-b4ea-339235d0fc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:44.781991 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0340f67-04cb-48dd-b4ea-339235d0fc8a admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:44.782318 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0340f67-04cb-48dd-b4ea-339235d0fc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:44.783241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0340f67-04cb-48dd-b4ea-339235d0fc8a admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:44.784455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0340f67-04cb-48dd-b4ea-339235d0fc8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:44.789919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0340f67-04cb-48dd-b4ea-339235d0fc8a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:44.828176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1003/1985] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:44 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:44.832120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae73aa1-7289-4fc4-a400-c64fc03a5901 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:44.847512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae73aa1-7289-4fc4-a400-c64fc03a5901 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:44.923319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:44.958953 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6a29a-b809-4038-962e-f801b10de8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:44.961470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6a29a-b809-4038-962e-f801b10de8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:44.961985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6a29a-b809-4038-962e-f801b10de8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:44.962560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6a29a-b809-4038-962e-f801b10de8d9 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:44.963517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6a29a-b809-4038-962e-f801b10de8d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:44.979706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6a29a-b809-4038-962e-f801b10de8d9 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:45.034178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 983/1986] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:44 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:45.035477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0340f67-04cb-48dd-b4ea-339235d0fc8a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:45.049291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0340f67-04cb-48dd-b4ea-339235d0fc8a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:45.056443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737511e2-f60d-403c-aa13-afa6a9bb1b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:45.057172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737511e2-f60d-403c-aa13-afa6a9bb1b2b admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.057517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737511e2-f60d-403c-aa13-afa6a9bb1b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.058001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737511e2-f60d-403c-aa13-afa6a9bb1b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.058719 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737511e2-f60d-403c-aa13-afa6a9bb1b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:45.060780 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737511e2-f60d-403c-aa13-afa6a9bb1b2b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:45.075445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1004/1987] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:46:45.076784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6a29a-b809-4038-962e-f801b10de8d9 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:45.108887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6a29a-b809-4038-962e-f801b10de8d9 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:45.137262 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233eaceb-b4ef-444c-9118-b0c0e75e7297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:45.138773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eaceb-b4ef-444c-9118-b0c0e75e7297 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.139890 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eaceb-b4ef-444c-9118-b0c0e75e7297 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.140725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eaceb-b4ef-444c-9118-b0c0e75e7297 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.141753 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233eaceb-b4ef-444c-9118-b0c0e75e7297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:45.144867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233eaceb-b4ef-444c-9118-b0c0e75e7297 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:45.158401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 984/1988] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:45.162178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737511e2-f60d-403c-aa13-afa6a9bb1b2b admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:45.177035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737511e2-f60d-403c-aa13-afa6a9bb1b2b admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:45.184482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528e7f7-4130-4a6a-9bbf-dfb8938aa9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:45.186063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528e7f7-4130-4a6a-9bbf-dfb8938aa9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.186357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528e7f7-4130-4a6a-9bbf-dfb8938aa9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.186874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528e7f7-4130-4a6a-9bbf-dfb8938aa9a2 admin admin] PATH_INFO: `/v3/projects/d2eb0b7f73c7417c91b72ae5446b0605/users/6e01098d9feb43bca25af1c5e118ec44/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.205261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:45.210868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528e7f7-4130-4a6a-9bbf-dfb8938aa9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2eb0b7f73c7417c91b72ae5446b0605, user_id=6e01098d9feb43bca25af1c5e118ec44, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:45.213920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528e7f7-4130-4a6a-9bbf-dfb8938aa9a2 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:45.237581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1005/1989] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:45 2022] PUT /identity/v3/projects/d2eb0b7f73c7417c91b72ae5446b0605/users/6e01098d9feb43bca25af1c5e118ec44/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:45.251696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bb1d9-7374-414d-b555-7e71ca9c30a2 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.252393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bb1d9-7374-414d-b555-7e71ca9c30a2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.252393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bb1d9-7374-414d-b555-7e71ca9c30a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.286533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-fd4bb1d9-7374-414d-b555-7e71ca9c30a2 None None] MFA Rules not processed for user `6e01098d9feb43bca25af1c5e118ec44`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:45.287883 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-fd4bb1d9-7374-414d-b555-7e71ca9c30a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:45.391661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 985/1990] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:45.396474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528e7f7-4130-4a6a-9bbf-dfb8938aa9a2 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:45.471464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528e7f7-4130-4a6a-9bbf-dfb8938aa9a2 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:45.479301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878532ff-601c-40e4-b2d5-1db5eaf39f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:45.479925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878532ff-601c-40e4-b2d5-1db5eaf39f9d admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.480245 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878532ff-601c-40e4-b2d5-1db5eaf39f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.480530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878532ff-601c-40e4-b2d5-1db5eaf39f9d admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.481223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878532ff-601c-40e4-b2d5-1db5eaf39f9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:45.483222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878532ff-601c-40e4-b2d5-1db5eaf39f9d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:45.511565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1006/1991] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:45 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:45.515670 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4bb1d9-7374-414d-b555-7e71ca9c30a2 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:45.526877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4bb1d9-7374-414d-b555-7e71ca9c30a2 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:45.607923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f1b51-6d2b-40fe-a4d0-c0105260ffbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:45.608899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f1b51-6d2b-40fe-a4d0-c0105260ffbc admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.609438 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f1b51-6d2b-40fe-a4d0-c0105260ffbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.610135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f1b51-6d2b-40fe-a4d0-c0105260ffbc admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.610972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f1b51-6d2b-40fe-a4d0-c0105260ffbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:45.613651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f1b51-6d2b-40fe-a4d0-c0105260ffbc admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:45.637381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 986/1992] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:45 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:45.642105 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878532ff-601c-40e4-b2d5-1db5eaf39f9d admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:45.654112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878532ff-601c-40e4-b2d5-1db5eaf39f9d admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:45.658670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0420cd6-6d77-4350-82f4-3428e1eebfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:45.659411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0420cd6-6d77-4350-82f4-3428e1eebfcd admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.659742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0420cd6-6d77-4350-82f4-3428e1eebfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.660079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0420cd6-6d77-4350-82f4-3428e1eebfcd admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.660790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0420cd6-6d77-4350-82f4-3428e1eebfcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:45.662448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0420cd6-6d77-4350-82f4-3428e1eebfcd admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:45.675239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1007/1993] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:46:45.679585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f1b51-6d2b-40fe-a4d0-c0105260ffbc admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:45.689896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f1b51-6d2b-40fe-a4d0-c0105260ffbc admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:45.694645 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6163819-2aa5-4ab7-8c55-032c57080d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:45.695372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6163819-2aa5-4ab7-8c55-032c57080d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.695667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6163819-2aa5-4ab7-8c55-032c57080d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.695944 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6163819-2aa5-4ab7-8c55-032c57080d90 admin admin] PATH_INFO: `/v3/projects/90d439eb7cfe480caebac6d082456d51/users/596b716275574323a251806c5122261a/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.711445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6163819-2aa5-4ab7-8c55-032c57080d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90d439eb7cfe480caebac6d082456d51, user_id=596b716275574323a251806c5122261a, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:45.714091 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6163819-2aa5-4ab7-8c55-032c57080d90 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:45.728221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 987/1994] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:45 2022] PUT /identity/v3/projects/90d439eb7cfe480caebac6d082456d51/users/596b716275574323a251806c5122261a/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:45.732308 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0420cd6-6d77-4350-82f4-3428e1eebfcd admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:45.742497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0420cd6-6d77-4350-82f4-3428e1eebfcd admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:45.801154 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8694cd9-9edb-4ebd-b0e5-c6d56faa379d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:45.801800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8694cd9-9edb-4ebd-b0e5-c6d56faa379d admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.802084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8694cd9-9edb-4ebd-b0e5-c6d56faa379d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.802359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8694cd9-9edb-4ebd-b0e5-c6d56faa379d admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.803004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8694cd9-9edb-4ebd-b0e5-c6d56faa379d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:45.804643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8694cd9-9edb-4ebd-b0e5-c6d56faa379d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:45.814401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1008/1995] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:46:45.818612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6163819-2aa5-4ab7-8c55-032c57080d90 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:45.828838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6163819-2aa5-4ab7-8c55-032c57080d90 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:45.833385 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f8d30-f0b4-4735-9da6-c4945d926393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:45.834069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f8d30-f0b4-4735-9da6-c4945d926393 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.834353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f8d30-f0b4-4735-9da6-c4945d926393 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.835184 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f8d30-f0b4-4735-9da6-c4945d926393 admin admin] PATH_INFO: `/v3/projects/90d439eb7cfe480caebac6d082456d51/users/596b716275574323a251806c5122261a/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.837202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f8d30-f0b4-4735-9da6-c4945d926393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90d439eb7cfe480caebac6d082456d51, user_id=596b716275574323a251806c5122261a, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:45.839895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f8d30-f0b4-4735-9da6-c4945d926393 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:45.851306 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 988/1996] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:45 2022] PUT /identity/v3/projects/90d439eb7cfe480caebac6d082456d51/users/596b716275574323a251806c5122261a/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:45.863530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33a622-4f9d-4e11-a0ad-ca9d22be4a4d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:45.863822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33a622-4f9d-4e11-a0ad-ca9d22be4a4d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:45.864671 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33a622-4f9d-4e11-a0ad-ca9d22be4a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:45.911747 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-3a33a622-4f9d-4e11-a0ad-ca9d22be4a4d None None] MFA Rules not processed for user `596b716275574323a251806c5122261a`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:45.913594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3a33a622-4f9d-4e11-a0ad-ca9d22be4a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:45.964633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:45.988235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1009/1997] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:45.998850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f8d30-f0b4-4735-9da6-c4945d926393 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:46.000591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e30f8d30-f0b4-4735-9da6-c4945d926393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:46.011727 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:46.060354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f8d30-f0b4-4735-9da6-c4945d926393 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:46.063954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82408e6a-de3a-43dd-89c0-828c636ee592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:46.064559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82408e6a-de3a-43dd-89c0-828c636ee592 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:46.064834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82408e6a-de3a-43dd-89c0-828c636ee592 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:46.065097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82408e6a-de3a-43dd-89c0-828c636ee592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:46.074382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82408e6a-de3a-43dd-89c0-828c636ee592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:46.075797 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82408e6a-de3a-43dd-89c0-828c636ee592 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:46.085051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 989/1998] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:46.187534 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33a622-4f9d-4e11-a0ad-ca9d22be4a4d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:46.197580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33a622-4f9d-4e11-a0ad-ca9d22be4a4d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:46.201209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df1e6a-af94-425d-9047-897332cb9b6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:46.201844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df1e6a-af94-425d-9047-897332cb9b6b service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:46.202127 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df1e6a-af94-425d-9047-897332cb9b6b service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:46.202401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df1e6a-af94-425d-9047-897332cb9b6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:46.278900 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df1e6a-af94-425d-9047-897332cb9b6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:46.280472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df1e6a-af94-425d-9047-897332cb9b6b service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:46.292660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1010/1999] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:46.472527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82408e6a-de3a-43dd-89c0-828c636ee592 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:46.473975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-82408e6a-de3a-43dd-89c0-828c636ee592 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:46.589412 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82408e6a-de3a-43dd-89c0-828c636ee592 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:46.594317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:46.595533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:46.595821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:46.596099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] PATH_INFO: `/v3/users/d08e5ace26c44f6b88f34cb213db28a4` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:46.597370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d08e5ace26c44f6b88f34cb213db28a4)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:46.599405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:46.676602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd08e5ace26c44f6b88f34cb213db28a4'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:46.677593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd08e5ace26c44f6b88f34cb213db28a4'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:46.702470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd08e5ace26c44f6b88f34cb213db28a4'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:46.712239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd08e5ace26c44f6b88f34cb213db28a4'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:46.713591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd08e5ace26c44f6b88f34cb213db28a4'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:46.732947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 990/2000] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:46 2022] DELETE /identity/v3/users/d08e5ace26c44f6b88f34cb213db28a4 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:46.737608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772e8d8-c756-4e3a-84fe-44bc94428cc5 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:46.738217 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772e8d8-c756-4e3a-84fe-44bc94428cc5 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:46.738217 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772e8d8-c756-4e3a-84fe-44bc94428cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:46.757439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b772e8d8-c756-4e3a-84fe-44bc94428cc5 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:46.758721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b772e8d8-c756-4e3a-84fe-44bc94428cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:46.815932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1011/2001] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:46.826109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:46.827498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:46.862205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:46.890595 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9db793-b3ba-4c67-9b1d-06230d85488a admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:46.899714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e01206-87b6-4f7e-b2c6-9f4d62d75149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:46.900793 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e01206-87b6-4f7e-b2c6-9f4d62d75149 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:46.901296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e01206-87b6-4f7e-b2c6-9f4d62d75149 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:46.901764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e01206-87b6-4f7e-b2c6-9f4d62d75149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:46.918484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e01206-87b6-4f7e-b2c6-9f4d62d75149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:46.920018 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e01206-87b6-4f7e-b2c6-9f4d62d75149 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:46.931756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 991/2002] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:46:47.261020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772e8d8-c756-4e3a-84fe-44bc94428cc5 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:47.263396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b772e8d8-c756-4e3a-84fe-44bc94428cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:47.276472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772e8d8-c756-4e3a-84fe-44bc94428cc5 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:47.281338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:47.282037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:47.282317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:47.282590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] PATH_INFO: `/v3/projects/f8169a00e0714eceb0d7337f691254f5` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:47.283717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8169a00e0714eceb0d7337f691254f5)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:47.285645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:47.373457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8169a00e0714eceb0d7337f691254f5', 'request_id': 'req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:47.380576 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8169a00e0714eceb0d7337f691254f5', 'request_id': 'req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:47.386641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8169a00e0714eceb0d7337f691254f5', 'request_id': 'req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:47.404480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] The token cache is being invalidate because project f8169a00e0714eceb0d7337f691254f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:47.405657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:47.406950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:47.412536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1012/2003] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:47 2022] DELETE /identity/v3/projects/f8169a00e0714eceb0d7337f691254f5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:47.414188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e01206-87b6-4f7e-b2c6-9f4d62d75149 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:47.430908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e01206-87b6-4f7e-b2c6-9f4d62d75149 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:47.500494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf802f2-503e-4241-8e1e-4e23876de12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:47.501133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf802f2-503e-4241-8e1e-4e23876de12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:47.501413 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf802f2-503e-4241-8e1e-4e23876de12b admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:47.501765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf802f2-503e-4241-8e1e-4e23876de12b admin admin] PATH_INFO: `/v3/users/a492e628fc7a47d7a4fb4f603995f55f` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:47.502836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf802f2-503e-4241-8e1e-4e23876de12b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a492e628fc7a47d7a4fb4f603995f55f)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:47.504776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf802f2-503e-4241-8e1e-4e23876de12b admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:47.549967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-dcf802f2-503e-4241-8e1e-4e23876de12b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a492e628fc7a47d7a4fb4f603995f55f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:47.550611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-dcf802f2-503e-4241-8e1e-4e23876de12b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a492e628fc7a47d7a4fb4f603995f55f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:47.563061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-dcf802f2-503e-4241-8e1e-4e23876de12b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a492e628fc7a47d7a4fb4f603995f55f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:47.570022 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-dcf802f2-503e-4241-8e1e-4e23876de12b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a492e628fc7a47d7a4fb4f603995f55f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:47.570693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-dcf802f2-503e-4241-8e1e-4e23876de12b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a492e628fc7a47d7a4fb4f603995f55f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:47.583195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 992/2004] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:47 2022] DELETE /identity/v3/users/a492e628fc7a47d7a4fb4f603995f55f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:47.973452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:47.975653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:47.998106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5710e3-e9b7-45ab-a3ec-fd5cc1ca21b6 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:48.012028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:48.122898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b9e96-a31a-4482-a04e-8958975a69c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:48.123741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b9e96-a31a-4482-a04e-8958975a69c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:48.124177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b9e96-a31a-4482-a04e-8958975a69c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:48.124414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b9e96-a31a-4482-a04e-8958975a69c4 admin admin] PATH_INFO: `/v3/projects/370c3ffafe3e44e0afa195511b2e63f5` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:48.126601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b9e96-a31a-4482-a04e-8958975a69c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=370c3ffafe3e44e0afa195511b2e63f5)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:48.128834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b9e96-a31a-4482-a04e-8958975a69c4 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:48.191570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-341b9e96-a31a-4482-a04e-8958975a69c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '370c3ffafe3e44e0afa195511b2e63f5', 'request_id': 'req-341b9e96-a31a-4482-a04e-8958975a69c4'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:48.198864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-341b9e96-a31a-4482-a04e-8958975a69c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '370c3ffafe3e44e0afa195511b2e63f5', 'request_id': 'req-341b9e96-a31a-4482-a04e-8958975a69c4'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:48.205096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-341b9e96-a31a-4482-a04e-8958975a69c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '370c3ffafe3e44e0afa195511b2e63f5', 'request_id': 'req-341b9e96-a31a-4482-a04e-8958975a69c4'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:48.219027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-341b9e96-a31a-4482-a04e-8958975a69c4 admin admin] The token cache is being invalidate because project 370c3ffafe3e44e0afa195511b2e63f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:48.219320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-341b9e96-a31a-4482-a04e-8958975a69c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:48.220234 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-341b9e96-a31a-4482-a04e-8958975a69c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:48.221863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1013/2005] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:47 2022] DELETE /identity/v3/projects/370c3ffafe3e44e0afa195511b2e63f5 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:48.227099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069df86-db11-4e74-898e-bf010378cb96 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:48.227389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069df86-db11-4e74-898e-bf010378cb96 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:48.227660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069df86-db11-4e74-898e-bf010378cb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:48.251299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-3069df86-db11-4e74-898e-bf010378cb96 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:48.252526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3069df86-db11-4e74-898e-bf010378cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:48.311108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:48.329390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 993/2006] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:48.338972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54e2ca-9aaf-4fee-a6e0-391d4847bea5 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:48.339540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54e2ca-9aaf-4fee-a6e0-391d4847bea5 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:48.339970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54e2ca-9aaf-4fee-a6e0-391d4847bea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:48.362731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-0a54e2ca-9aaf-4fee-a6e0-391d4847bea5 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:48.364120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-0a54e2ca-9aaf-4fee-a6e0-391d4847bea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:48.372229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1014/2007] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:48.376590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069df86-db11-4e74-898e-bf010378cb96 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:48.387818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069df86-db11-4e74-898e-bf010378cb96 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:48.392964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd6ef0-fdf8-4782-9f30-ad9bde213caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:48.393659 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd6ef0-fdf8-4782-9f30-ad9bde213caa admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:48.394075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd6ef0-fdf8-4782-9f30-ad9bde213caa admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:48.394420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd6ef0-fdf8-4782-9f30-ad9bde213caa admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:48.395145 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd6ef0-fdf8-4782-9f30-ad9bde213caa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:48.395539 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd6ef0-fdf8-4782-9f30-ad9bde213caa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:48.398556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd6ef0-fdf8-4782-9f30-ad9bde213caa admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:48.411449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 994/2008] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:46:48.416807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54e2ca-9aaf-4fee-a6e0-391d4847bea5 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:48.429026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54e2ca-9aaf-4fee-a6e0-391d4847bea5 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:48.445313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0651d-399b-4f2b-8a50-e3519642e803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:48.445944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0651d-399b-4f2b-8a50-e3519642e803 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:48.446232 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0651d-399b-4f2b-8a50-e3519642e803 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:48.446506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0651d-399b-4f2b-8a50-e3519642e803 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:48.447266 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0651d-399b-4f2b-8a50-e3519642e803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:48.449189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0651d-399b-4f2b-8a50-e3519642e803 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:48.507693 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1015/2009] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:48 2022] POST /identity/v3/projects => generated 359 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:48.511716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd6ef0-fdf8-4782-9f30-ad9bde213caa admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:48.531968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd6ef0-fdf8-4782-9f30-ad9bde213caa admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:48.600921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b4951-9490-4ac0-ae2a-e6988ba061eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:48.601965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b4951-9490-4ac0-ae2a-e6988ba061eb admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:48.602406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b4951-9490-4ac0-ae2a-e6988ba061eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:48.602831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b4951-9490-4ac0-ae2a-e6988ba061eb admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:48.603740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b4951-9490-4ac0-ae2a-e6988ba061eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:48.606756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b4951-9490-4ac0-ae2a-e6988ba061eb admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:48.642431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 995/2010] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:48 2022] POST /identity/v3/users => generated 280 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:46:48.643953 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0651d-399b-4f2b-8a50-e3519642e803 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:48.666814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0651d-399b-4f2b-8a50-e3519642e803 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:48.672693 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8260f56-365d-4222-b80f-a36979330cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:48.673230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8260f56-365d-4222-b80f-a36979330cda admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:48.673616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8260f56-365d-4222-b80f-a36979330cda admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:48.674010 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8260f56-365d-4222-b80f-a36979330cda admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:48.674770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8260f56-365d-4222-b80f-a36979330cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:48.676679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8260f56-365d-4222-b80f-a36979330cda admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:48.688600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1016/2011] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:48.692604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b4951-9490-4ac0-ae2a-e6988ba061eb admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:48.703656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b4951-9490-4ac0-ae2a-e6988ba061eb admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:48.708634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897c0d9-d121-4d2c-a289-ce6577058d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:48.709277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897c0d9-d121-4d2c-a289-ce6577058d98 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:48.709558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897c0d9-d121-4d2c-a289-ce6577058d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:48.709884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897c0d9-d121-4d2c-a289-ce6577058d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:48.710507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897c0d9-d121-4d2c-a289-ce6577058d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:48.712131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897c0d9-d121-4d2c-a289-ce6577058d98 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:48.728508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 996/2012] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:48.729770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8260f56-365d-4222-b80f-a36979330cda admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:48.739963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8260f56-365d-4222-b80f-a36979330cda admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:48.745515 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f42b83-fd68-4fa7-bb82-b90da7843a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:48.745995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f42b83-fd68-4fa7-bb82-b90da7843a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:48.746286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f42b83-fd68-4fa7-bb82-b90da7843a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:48.746564 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f42b83-fd68-4fa7-bb82-b90da7843a23 admin admin] PATH_INFO: `/v3/projects/e8f30ff339d9477aace89a85086c3fed/users/98cdb1d675c44108a3a472ad4cf8e778/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:48.764573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f42b83-fd68-4fa7-bb82-b90da7843a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f30ff339d9477aace89a85086c3fed, user_id=98cdb1d675c44108a3a472ad4cf8e778, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:48.767277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f42b83-fd68-4fa7-bb82-b90da7843a23 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:48.783183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1017/2013] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:48 2022] PUT /identity/v3/projects/e8f30ff339d9477aace89a85086c3fed/users/98cdb1d675c44108a3a472ad4cf8e778/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:48.794404 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1aeb8a-6315-4409-8771-358c1548116f None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:48.794691 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1aeb8a-6315-4409-8771-358c1548116f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:48.795034 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1aeb8a-6315-4409-8771-358c1548116f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:48.824851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-0e1aeb8a-6315-4409-8771-358c1548116f None None] MFA Rules not processed for user `98cdb1d675c44108a3a472ad4cf8e778`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:48.826296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0e1aeb8a-6315-4409-8771-358c1548116f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:48.921734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 997/2014] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:48.966978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f42b83-fd68-4fa7-bb82-b90da7843a23 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:48.969328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a1f42b83-fd68-4fa7-bb82-b90da7843a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:49.018567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:49.050214 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f42b83-fd68-4fa7-bb82-b90da7843a23 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:49.057468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303e1eb-788f-4271-94d5-508820c6f2c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:49.058394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303e1eb-788f-4271-94d5-508820c6f2c1 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:49.058872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303e1eb-788f-4271-94d5-508820c6f2c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:49.059438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303e1eb-788f-4271-94d5-508820c6f2c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:49.073331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303e1eb-788f-4271-94d5-508820c6f2c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:49.075250 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303e1eb-788f-4271-94d5-508820c6f2c1 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:49.089344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1018/2015] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:48 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:49.107531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1aeb8a-6315-4409-8771-358c1548116f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:49.171629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:49.175789 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303e1eb-788f-4271-94d5-508820c6f2c1 service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:49.180120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c303e1eb-788f-4271-94d5-508820c6f2c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:49.202741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1aeb8a-6315-4409-8771-358c1548116f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:49.208048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a631549-527b-45d2-a354-c35e29174163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:49.208688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631549-527b-45d2-a354-c35e29174163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:49.208963 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631549-527b-45d2-a354-c35e29174163 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:49.209263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631549-527b-45d2-a354-c35e29174163 admin admin] PATH_INFO: `/v3/users/98cdb1d675c44108a3a472ad4cf8e778` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:49.210584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a631549-527b-45d2-a354-c35e29174163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98cdb1d675c44108a3a472ad4cf8e778)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:49.212525 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a631549-527b-45d2-a354-c35e29174163 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:49.271684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9a631549-527b-45d2-a354-c35e29174163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98cdb1d675c44108a3a472ad4cf8e778'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:49.272977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9a631549-527b-45d2-a354-c35e29174163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98cdb1d675c44108a3a472ad4cf8e778'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:49.276715 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303e1eb-788f-4271-94d5-508820c6f2c1 service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:49.282491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1cb41-13c5-4294-b1a6-2620faa0a62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:49.283396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1cb41-13c5-4294-b1a6-2620faa0a62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:49.283687 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1cb41-13c5-4294-b1a6-2620faa0a62f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:49.283969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1cb41-13c5-4294-b1a6-2620faa0a62f admin admin] PATH_INFO: `/v3/users/6e01098d9feb43bca25af1c5e118ec44` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:49.285129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1cb41-13c5-4294-b1a6-2620faa0a62f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e01098d9feb43bca25af1c5e118ec44)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:49.287429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1cb41-13c5-4294-b1a6-2620faa0a62f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:49.287544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9a631549-527b-45d2-a354-c35e29174163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98cdb1d675c44108a3a472ad4cf8e778'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:49.295799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9a631549-527b-45d2-a354-c35e29174163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98cdb1d675c44108a3a472ad4cf8e778'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:49.296497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9a631549-527b-45d2-a354-c35e29174163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98cdb1d675c44108a3a472ad4cf8e778'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:49.315004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 998/2016] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:49 2022] DELETE /identity/v3/users/98cdb1d675c44108a3a472ad4cf8e778 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:49.316856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b78600-75b0-4eca-99b5-7e722c41dd2a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:49.317530 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b78600-75b0-4eca-99b5-7e722c41dd2a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:49.317530 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b78600-75b0-4eca-99b5-7e722c41dd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:49.342178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-23b78600-75b0-4eca-99b5-7e722c41dd2a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:49.343294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-23b78600-75b0-4eca-99b5-7e722c41dd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:49.366437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e4e1cb41-13c5-4294-b1a6-2620faa0a62f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e01098d9feb43bca25af1c5e118ec44'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:49.367545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e4e1cb41-13c5-4294-b1a6-2620faa0a62f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e01098d9feb43bca25af1c5e118ec44'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:49.375057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e4e1cb41-13c5-4294-b1a6-2620faa0a62f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e01098d9feb43bca25af1c5e118ec44'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:49.375651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e4e1cb41-13c5-4294-b1a6-2620faa0a62f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e01098d9feb43bca25af1c5e118ec44'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:49.390911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e4e1cb41-13c5-4294-b1a6-2620faa0a62f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e01098d9feb43bca25af1c5e118ec44'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:49.407760 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1019/2017] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:49 2022] DELETE /identity/v3/users/6e01098d9feb43bca25af1c5e118ec44 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:49.411758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f0202-1395-4d72-b79e-639d8fea00ef None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:49.412467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f0202-1395-4d72-b79e-639d8fea00ef None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:49.412467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f0202-1395-4d72-b79e-639d8fea00ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:49.422576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 999/2018] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:49.432968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b78600-75b0-4eca-99b5-7e722c41dd2a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:49.434369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-505f0202-1395-4d72-b79e-639d8fea00ef None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:49.434771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-23b78600-75b0-4eca-99b5-7e722c41dd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:49.436174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-505f0202-1395-4d72-b79e-639d8fea00ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:49.507897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:49.515377 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b78600-75b0-4eca-99b5-7e722c41dd2a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:49.525984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1020/2019] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:49.545404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f0202-1395-4d72-b79e-639d8fea00ef None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:49.557925 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:49.617732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1f940-90a1-4ad3-946e-9ae59f99038d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:49.617732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1f940-90a1-4ad3-946e-9ae59f99038d service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:49.617732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1f940-90a1-4ad3-946e-9ae59f99038d service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:49.617732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1f940-90a1-4ad3-946e-9ae59f99038d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:49.621733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f0202-1395-4d72-b79e-639d8fea00ef None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:49.621944 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1f940-90a1-4ad3-946e-9ae59f99038d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:49.623477 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1f940-90a1-4ad3-946e-9ae59f99038d service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:49.624312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e920b-4d15-4aa8-94b4-c0d05dc105df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:49.624923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e920b-4d15-4aa8-94b4-c0d05dc105df service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:49.625206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e920b-4d15-4aa8-94b4-c0d05dc105df service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:49.625482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e920b-4d15-4aa8-94b4-c0d05dc105df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:49.634989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1000/2020] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:46:49.638470 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e920b-4d15-4aa8-94b4-c0d05dc105df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:49.639989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e920b-4d15-4aa8-94b4-c0d05dc105df service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:49.651620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1021/2021] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:46:50.221324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1f940-90a1-4ad3-946e-9ae59f99038d service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:50.222811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-19e1f940-90a1-4ad3-946e-9ae59f99038d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:50.252242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e920b-4d15-4aa8-94b4-c0d05dc105df service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:50.253848 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-067e920b-4d15-4aa8-94b4-c0d05dc105df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:50.259617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1f940-90a1-4ad3-946e-9ae59f99038d service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:50.266028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b350698-8e27-4595-b840-530f69401535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:50.266975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b350698-8e27-4595-b840-530f69401535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:50.267356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b350698-8e27-4595-b840-530f69401535 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:50.267630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b350698-8e27-4595-b840-530f69401535 admin admin] PATH_INFO: `/v3/projects/e8f30ff339d9477aace89a85086c3fed` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:50.268903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b350698-8e27-4595-b840-530f69401535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8f30ff339d9477aace89a85086c3fed)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:50.272034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e920b-4d15-4aa8-94b4-c0d05dc105df service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:50.282352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b350698-8e27-4595-b840-530f69401535 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:50.282637 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:50.286110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:50.286443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:50.286875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] PATH_INFO: `/v3/projects/d2eb0b7f73c7417c91b72ae5446b0605` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:50.288497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2eb0b7f73c7417c91b72ae5446b0605)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:50.290707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:50.372118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2eb0b7f73c7417c91b72ae5446b0605', 'request_id': 'req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.384115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6b350698-8e27-4595-b840-530f69401535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8f30ff339d9477aace89a85086c3fed', 'request_id': 'req-6b350698-8e27-4595-b840-530f69401535'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.390126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2eb0b7f73c7417c91b72ae5446b0605', 'request_id': 'req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.399825 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6b350698-8e27-4595-b840-530f69401535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8f30ff339d9477aace89a85086c3fed', 'request_id': 'req-6b350698-8e27-4595-b840-530f69401535'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.404187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2eb0b7f73c7417c91b72ae5446b0605', 'request_id': 'req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.415495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6b350698-8e27-4595-b840-530f69401535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8f30ff339d9477aace89a85086c3fed', 'request_id': 'req-6b350698-8e27-4595-b840-530f69401535'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.427759 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] The token cache is being invalidate because project d2eb0b7f73c7417c91b72ae5446b0605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:50.428715 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.429466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.432225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1022/2022] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:50 2022] DELETE /identity/v3/projects/d2eb0b7f73c7417c91b72ae5446b0605 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:50.436115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:50.437580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:50.438995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6b350698-8e27-4595-b840-530f69401535 admin admin] The token cache is being invalidate because project e8f30ff339d9477aace89a85086c3fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:50.440217 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6b350698-8e27-4595-b840-530f69401535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.441947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6b350698-8e27-4595-b840-530f69401535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.444549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1001/2023] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:50 2022] DELETE /identity/v3/projects/e8f30ff339d9477aace89a85086c3fed => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:50.449831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce6924-3afd-4981-88d2-e03e56020d6e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:50.450879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce6924-3afd-4981-88d2-e03e56020d6e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:50.451169 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce6924-3afd-4981-88d2-e03e56020d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:50.476695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-2bce6924-3afd-4981-88d2-e03e56020d6e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:50.477971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2bce6924-3afd-4981-88d2-e03e56020d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:50.526700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb316bba-fb2d-4a11-ab1d-f60ba0f89bd9 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:50.555921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:50.591261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1002/2024] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:50.599653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7caa6-cc4b-4654-be58-7274da5e8569 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:50.600440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7caa6-cc4b-4654-be58-7274da5e8569 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:50.600748 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7caa6-cc4b-4654-be58-7274da5e8569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:50.600978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:50.601723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:50.601926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:50.602198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] PATH_INFO: `/v3/users/596b716275574323a251806c5122261a` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:50.603504 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] RBAC: Authorizing `identity:delete_user(user_id=596b716275574323a251806c5122261a)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:50.605566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:50.627119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-9de7caa6-cc4b-4654-be58-7274da5e8569 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:50.628459 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9de7caa6-cc4b-4654-be58-7274da5e8569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:50.640400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1003/2025] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:46:50.646707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7caa6-cc4b-4654-be58-7274da5e8569 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:50.669667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7caa6-cc4b-4654-be58-7274da5e8569 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:50.674161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '596b716275574323a251806c5122261a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.675769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '596b716275574323a251806c5122261a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.675912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406fd0d-84fb-441b-b63d-301134de6e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:50.676706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406fd0d-84fb-441b-b63d-301134de6e3f admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:50.676989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406fd0d-84fb-441b-b63d-301134de6e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:50.677400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406fd0d-84fb-441b-b63d-301134de6e3f admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:50.678808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406fd0d-84fb-441b-b63d-301134de6e3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:50.679117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406fd0d-84fb-441b-b63d-301134de6e3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:50.680865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406fd0d-84fb-441b-b63d-301134de6e3f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:50.686170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '596b716275574323a251806c5122261a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.687107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '596b716275574323a251806c5122261a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.702958 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '596b716275574323a251806c5122261a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:50.717537 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1004/2026] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:46:50.722072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406fd0d-84fb-441b-b63d-301134de6e3f admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:50.723826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4406fd0d-84fb-441b-b63d-301134de6e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:50.740244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406fd0d-84fb-441b-b63d-301134de6e3f admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:50.741353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1023/2027] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:46:50 2022] DELETE /identity/v3/users/596b716275574323a251806c5122261a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:50.776836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:50.816419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6df87d-9867-49ac-953c-fd53bec58691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:50.817105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6df87d-9867-49ac-953c-fd53bec58691 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:50.817607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6df87d-9867-49ac-953c-fd53bec58691 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:50.818381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6df87d-9867-49ac-953c-fd53bec58691 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:50.820797 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6df87d-9867-49ac-953c-fd53bec58691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:50.826707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6df87d-9867-49ac-953c-fd53bec58691 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:50.891380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1005/2028] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:50 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:50.895472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:50.908363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104cb87e-505d-4b37-a7a8-31e236f8a90b admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:50.977080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd18ce7-f6b0-471d-a505-018a1bb6a8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:50.977997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd18ce7-f6b0-471d-a505-018a1bb6a8ba admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:50.978288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd18ce7-f6b0-471d-a505-018a1bb6a8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:50.979098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd18ce7-f6b0-471d-a505-018a1bb6a8ba admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:50.979749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd18ce7-f6b0-471d-a505-018a1bb6a8ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:50.982607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd18ce7-f6b0-471d-a505-018a1bb6a8ba admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:51.012327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1024/2029] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:50 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:51.016525 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6df87d-9867-49ac-953c-fd53bec58691 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:51.027589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6df87d-9867-49ac-953c-fd53bec58691 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:51.033723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f780dd2-6aae-44d1-a641-11af60e19334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:51.034439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f780dd2-6aae-44d1-a641-11af60e19334 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:51.035187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f780dd2-6aae-44d1-a641-11af60e19334 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:51.035187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f780dd2-6aae-44d1-a641-11af60e19334 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:51.048779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f780dd2-6aae-44d1-a641-11af60e19334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:51.051051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f780dd2-6aae-44d1-a641-11af60e19334 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:51.063916 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1006/2030] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:51.068446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd18ce7-f6b0-471d-a505-018a1bb6a8ba admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:51.080908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd18ce7-f6b0-471d-a505-018a1bb6a8ba admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:51.088674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00071d6e-4f55-4567-8f5b-f462aa2d26a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:51.089295 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00071d6e-4f55-4567-8f5b-f462aa2d26a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:51.089574 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00071d6e-4f55-4567-8f5b-f462aa2d26a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:51.090472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00071d6e-4f55-4567-8f5b-f462aa2d26a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:51.091140 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00071d6e-4f55-4567-8f5b-f462aa2d26a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:51.092799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00071d6e-4f55-4567-8f5b-f462aa2d26a6 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:51.112918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1025/2031] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:51.114308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f780dd2-6aae-44d1-a641-11af60e19334 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:51.124885 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f780dd2-6aae-44d1-a641-11af60e19334 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:51.129766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601094a-5b49-474b-a470-42a4297c1060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:51.130371 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601094a-5b49-474b-a470-42a4297c1060 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:51.130666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601094a-5b49-474b-a470-42a4297c1060 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:51.131074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601094a-5b49-474b-a470-42a4297c1060 admin admin] PATH_INFO: `/v3/projects/805addb87ebc4c08ab117be25b17b6de/users/4eb0674c8b274fa29908ac76842e7a4d/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:51.147979 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601094a-5b49-474b-a470-42a4297c1060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=805addb87ebc4c08ab117be25b17b6de, user_id=4eb0674c8b274fa29908ac76842e7a4d, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:51.150694 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601094a-5b49-474b-a470-42a4297c1060 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:51.166941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1007/2032] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:51 2022] PUT /identity/v3/projects/805addb87ebc4c08ab117be25b17b6de/users/4eb0674c8b274fa29908ac76842e7a4d/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:51.177443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c273a73-f5f9-4434-b908-da87ce91c231 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:51.177762 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c273a73-f5f9-4434-b908-da87ce91c231 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:51.178114 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c273a73-f5f9-4434-b908-da87ce91c231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:51.190729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601094a-5b49-474b-a470-42a4297c1060 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:51.191923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9601094a-5b49-474b-a470-42a4297c1060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:51.225595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-1c273a73-f5f9-4434-b908-da87ce91c231 None None] MFA Rules not processed for user `4eb0674c8b274fa29908ac76842e7a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:51.227140 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1c273a73-f5f9-4434-b908-da87ce91c231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:51.252553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:51.317782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601094a-5b49-474b-a470-42a4297c1060 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:51.327102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:51.328182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:51.328638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:51.329081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d admin admin] PATH_INFO: `/v3/projects/90d439eb7cfe480caebac6d082456d51` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:51.330886 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=90d439eb7cfe480caebac6d082456d51)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:51.333653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:51.355244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1026/2033] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:51.367876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c273a73-f5f9-4434-b908-da87ce91c231 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:51.369406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1c273a73-f5f9-4434-b908-da87ce91c231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:51.417355 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90d439eb7cfe480caebac6d082456d51', 'request_id': 'req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:51.425236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90d439eb7cfe480caebac6d082456d51', 'request_id': 'req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:51.432128 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90d439eb7cfe480caebac6d082456d51', 'request_id': 'req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:51.442034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c273a73-f5f9-4434-b908-da87ce91c231 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:51.451044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d admin admin] The token cache is being invalidate because project 90d439eb7cfe480caebac6d082456d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:46:51.451437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:51.452366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-eaa558df-d704-4ddd-a3b3-0c82132b2a0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:46:51.454407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1008/2034] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:46:51 2022] DELETE /identity/v3/projects/90d439eb7cfe480caebac6d082456d51 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:51.469938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508e182-5f08-43ed-b694-453231fa9bef None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:51.470243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508e182-5f08-43ed-b694-453231fa9bef None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:51.470497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508e182-5f08-43ed-b694-453231fa9bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:51.495212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c11eb-5999-4f90-9cd6-ef4bbee6f606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:51.495831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c11eb-5999-4f90-9cd6-ef4bbee6f606 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:51.496118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c11eb-5999-4f90-9cd6-ef4bbee6f606 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:51.496386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c11eb-5999-4f90-9cd6-ef4bbee6f606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:51.497622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a508e182-5f08-43ed-b694-453231fa9bef None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:51.498338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-636c11eb-5999-4f90-9cd6-ef4bbee6f606 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:51.499049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a508e182-5f08-43ed-b694-453231fa9bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:51.507734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:51.596357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c11eb-5999-4f90-9cd6-ef4bbee6f606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:51.599649 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c11eb-5999-4f90-9cd6-ef4bbee6f606 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:51.637636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1027/2035] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:46:51.648417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1009/2036] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:51.653260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a806252-3e71-4d52-8891-d8e5d443f761 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:51.653260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a806252-3e71-4d52-8891-d8e5d443f761 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:51.653260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a806252-3e71-4d52-8891-d8e5d443f761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:51.677679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6a806252-3e71-4d52-8891-d8e5d443f761 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:51.679556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6a806252-3e71-4d52-8891-d8e5d443f761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:51.694531 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1028/2037] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:51.694633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508e182-5f08-43ed-b694-453231fa9bef None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:51.705001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508e182-5f08-43ed-b694-453231fa9bef None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:51.711410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d356864-ab87-40e5-9f4e-7ffa3f19a6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:51.712336 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d356864-ab87-40e5-9f4e-7ffa3f19a6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:51.712718 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d356864-ab87-40e5-9f4e-7ffa3f19a6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:51.713093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d356864-ab87-40e5-9f4e-7ffa3f19a6d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:51.714067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d356864-ab87-40e5-9f4e-7ffa3f19a6d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:46:51.714476 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d356864-ab87-40e5-9f4e-7ffa3f19a6d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:51.716174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d356864-ab87-40e5-9f4e-7ffa3f19a6d8 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:51.730985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1010/2038] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:46:51.735233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a806252-3e71-4d52-8891-d8e5d443f761 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:51.747951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a806252-3e71-4d52-8891-d8e5d443f761 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:51.754543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de29db3-a91d-4a51-9d59-e9a8f007f674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:51.755761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de29db3-a91d-4a51-9d59-e9a8f007f674 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:51.756619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de29db3-a91d-4a51-9d59-e9a8f007f674 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:51.757036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de29db3-a91d-4a51-9d59-e9a8f007f674 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:51.757871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de29db3-a91d-4a51-9d59-e9a8f007f674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:51.760951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de29db3-a91d-4a51-9d59-e9a8f007f674 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:51.798623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:51.808341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1029/2039] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:51 2022] POST /identity/v3/projects => generated 409 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:51.812590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d356864-ab87-40e5-9f4e-7ffa3f19a6d8 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:51.825746 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d356864-ab87-40e5-9f4e-7ffa3f19a6d8 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:51.902825 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c38a7a-b409-4605-8eb4-87eded70cbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:51.903448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c38a7a-b409-4605-8eb4-87eded70cbae admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:51.903730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c38a7a-b409-4605-8eb4-87eded70cbae admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:51.904027 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c38a7a-b409-4605-8eb4-87eded70cbae admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:51.904674 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c38a7a-b409-4605-8eb4-87eded70cbae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:51.906634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c38a7a-b409-4605-8eb4-87eded70cbae admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:51.941650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1011/2040] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:51 2022] POST /identity/v3/users => generated 305 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:51.942679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de29db3-a91d-4a51-9d59-e9a8f007f674 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:51.955154 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de29db3-a91d-4a51-9d59-e9a8f007f674 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:51.967547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ecd3a-7ad2-472b-9c3a-4514f5c06ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:51.968202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ecd3a-7ad2-472b-9c3a-4514f5c06ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:51.968447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ecd3a-7ad2-472b-9c3a-4514f5c06ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:51.968791 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ecd3a-7ad2-472b-9c3a-4514f5c06ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:51.969792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ecd3a-7ad2-472b-9c3a-4514f5c06ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:51.972561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ecd3a-7ad2-472b-9c3a-4514f5c06ee7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:51.990132 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1030/2041] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:51.993870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c38a7a-b409-4605-8eb4-87eded70cbae admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:52.004638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c38a7a-b409-4605-8eb4-87eded70cbae admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:52.014203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d158b-2fd1-4bd1-8e4c-a378cdce9736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:52.015042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d158b-2fd1-4bd1-8e4c-a378cdce9736 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:52.015466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d158b-2fd1-4bd1-8e4c-a378cdce9736 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:52.015797 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d158b-2fd1-4bd1-8e4c-a378cdce9736 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:52.016590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d158b-2fd1-4bd1-8e4c-a378cdce9736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:52.023226 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d158b-2fd1-4bd1-8e4c-a378cdce9736 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:52.043647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1012/2042] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:46:52.047541 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ecd3a-7ad2-472b-9c3a-4514f5c06ee7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:52.059257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ecd3a-7ad2-472b-9c3a-4514f5c06ee7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:52.064311 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5646cb6-5a9d-4d8f-b2a6-5c09129d3add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:52.064967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5646cb6-5a9d-4d8f-b2a6-5c09129d3add admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:52.065247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5646cb6-5a9d-4d8f-b2a6-5c09129d3add admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:52.065526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5646cb6-5a9d-4d8f-b2a6-5c09129d3add admin admin] PATH_INFO: `/v3/projects/722ff48d7acd47e8a7d696d0f5fa732f/users/b4f7a8d8ffdd42a7a51be6ce15f1a982/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:52.081926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5646cb6-5a9d-4d8f-b2a6-5c09129d3add admin admin] RBAC: Authorizing `identity:create_grant(project_id=722ff48d7acd47e8a7d696d0f5fa732f, user_id=b4f7a8d8ffdd42a7a51be6ce15f1a982, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:52.085244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5646cb6-5a9d-4d8f-b2a6-5c09129d3add admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:52.098322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1031/2043] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:52 2022] PUT /identity/v3/projects/722ff48d7acd47e8a7d696d0f5fa732f/users/b4f7a8d8ffdd42a7a51be6ce15f1a982/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:52.108483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac523aa-ba30-4612-a021-7b77f47f6744 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:52.108787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac523aa-ba30-4612-a021-7b77f47f6744 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:52.109064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac523aa-ba30-4612-a021-7b77f47f6744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:52.136657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-cac523aa-ba30-4612-a021-7b77f47f6744 None None] MFA Rules not processed for user `b4f7a8d8ffdd42a7a51be6ce15f1a982`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:52.137818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-cac523aa-ba30-4612-a021-7b77f47f6744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:52.202785 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1013/2044] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:52 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:52.207295 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5646cb6-5a9d-4d8f-b2a6-5c09129d3add admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:52.271526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5646cb6-5a9d-4d8f-b2a6-5c09129d3add admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:52.276369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7534f2a-ade2-45ec-bcf0-3149ba22db72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:52.277533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7534f2a-ade2-45ec-bcf0-3149ba22db72 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:52.277852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7534f2a-ade2-45ec-bcf0-3149ba22db72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:52.278124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7534f2a-ade2-45ec-bcf0-3149ba22db72 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:52.278765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7534f2a-ade2-45ec-bcf0-3149ba22db72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:52.280682 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7534f2a-ade2-45ec-bcf0-3149ba22db72 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:52.307207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1032/2045] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:46:52 2022] POST /identity/v3/projects => generated 407 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:52.311334 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac523aa-ba30-4612-a021-7b77f47f6744 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:52.322947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac523aa-ba30-4612-a021-7b77f47f6744 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:52.326709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:52.382325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf1e0b-effc-4e3b-84c6-e30689b69273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:52.383000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf1e0b-effc-4e3b-84c6-e30689b69273 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:52.383322 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf1e0b-effc-4e3b-84c6-e30689b69273 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:52.383607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf1e0b-effc-4e3b-84c6-e30689b69273 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:52.384250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf1e0b-effc-4e3b-84c6-e30689b69273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:52.386194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf1e0b-effc-4e3b-84c6-e30689b69273 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:52.408963 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1014/2046] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:46:52 2022] POST /identity/v3/users => generated 310 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:46:52.412784 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7534f2a-ade2-45ec-bcf0-3149ba22db72 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:52.423712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7534f2a-ade2-45ec-bcf0-3149ba22db72 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:52.429142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b1dc3-b1f9-4e1b-9f32-ca6b8953f290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:52.429772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b1dc3-b1f9-4e1b-9f32-ca6b8953f290 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:52.430056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b1dc3-b1f9-4e1b-9f32-ca6b8953f290 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:52.430743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b1dc3-b1f9-4e1b-9f32-ca6b8953f290 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:52.431026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b1dc3-b1f9-4e1b-9f32-ca6b8953f290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:52.432675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b1dc3-b1f9-4e1b-9f32-ca6b8953f290 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:52.447138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1033/2047] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:46:52.448401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf1e0b-effc-4e3b-84c6-e30689b69273 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:52.458563 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf1e0b-effc-4e3b-84c6-e30689b69273 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:52.463576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c838512-f7fa-4867-b692-7d78d2e2bb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:52.464263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c838512-f7fa-4867-b692-7d78d2e2bb43 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:52.464550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c838512-f7fa-4867-b692-7d78d2e2bb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:52.464827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c838512-f7fa-4867-b692-7d78d2e2bb43 admin admin] PATH_INFO: `/v3/projects/ef32c086b039433eb497f6a8e8ccfbd1/users/1a78de9468ed4c80a8926eb45b6d6e78/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:52.483888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c838512-f7fa-4867-b692-7d78d2e2bb43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef32c086b039433eb497f6a8e8ccfbd1, user_id=1a78de9468ed4c80a8926eb45b6d6e78, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:52.487126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c838512-f7fa-4867-b692-7d78d2e2bb43 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:52.504820 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1015/2048] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:52 2022] PUT /identity/v3/projects/ef32c086b039433eb497f6a8e8ccfbd1/users/1a78de9468ed4c80a8926eb45b6d6e78/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:46:52.509233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b1dc3-b1f9-4e1b-9f32-ca6b8953f290 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:52.521866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b1dc3-b1f9-4e1b-9f32-ca6b8953f290 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:52.582262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64c150-814e-473b-be06-8ef154d889e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:52.582878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64c150-814e-473b-be06-8ef154d889e8 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:52.583164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64c150-814e-473b-be06-8ef154d889e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:52.583443 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64c150-814e-473b-be06-8ef154d889e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:52.584622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64c150-814e-473b-be06-8ef154d889e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:52.586263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64c150-814e-473b-be06-8ef154d889e8 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:52.601117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1034/2049] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:46:52.602313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c838512-f7fa-4867-b692-7d78d2e2bb43 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:52.613071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c838512-f7fa-4867-b692-7d78d2e2bb43 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:52.618022 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a3474-420e-4d6e-a5c5-16c6659ac6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:52.618796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a3474-420e-4d6e-a5c5-16c6659ac6f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:52.619153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a3474-420e-4d6e-a5c5-16c6659ac6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:52.619489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a3474-420e-4d6e-a5c5-16c6659ac6f0 admin admin] PATH_INFO: `/v3/projects/ef32c086b039433eb497f6a8e8ccfbd1/users/1a78de9468ed4c80a8926eb45b6d6e78/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:52.621493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a3474-420e-4d6e-a5c5-16c6659ac6f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef32c086b039433eb497f6a8e8ccfbd1, user_id=1a78de9468ed4c80a8926eb45b6d6e78, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:52.624318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a3474-420e-4d6e-a5c5-16c6659ac6f0 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:52.635623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1016/2050] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:46:52 2022] PUT /identity/v3/projects/ef32c086b039433eb497f6a8e8ccfbd1/users/1a78de9468ed4c80a8926eb45b6d6e78/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:46:52.647622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1a5f4-7f5b-46ac-9e52-91b37934e778 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:52.647912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1a5f4-7f5b-46ac-9e52-91b37934e778 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:52.648189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1a5f4-7f5b-46ac-9e52-91b37934e778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:52.685343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d2e1a5f4-7f5b-46ac-9e52-91b37934e778 None None] MFA Rules not processed for user `1a78de9468ed4c80a8926eb45b6d6e78`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:46:52.686680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d2e1a5f4-7f5b-46ac-9e52-91b37934e778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:52.715854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:46:52.783887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1035/2051] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:46:52 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:46:52.794808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a3474-420e-4d6e-a5c5-16c6659ac6f0 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:52.796552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-264a3474-420e-4d6e-a5c5-16c6659ac6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:52.857281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a3474-420e-4d6e-a5c5-16c6659ac6f0 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:52.861426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4760af-9474-48eb-bddd-debbf48e00e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:52.862191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4760af-9474-48eb-bddd-debbf48e00e3 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:52.862611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4760af-9474-48eb-bddd-debbf48e00e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:52.862894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4760af-9474-48eb-bddd-debbf48e00e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:52.872139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4760af-9474-48eb-bddd-debbf48e00e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:52.873604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4760af-9474-48eb-bddd-debbf48e00e3 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:52.883463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1017/2052] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:52 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:46:53.057979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1a5f4-7f5b-46ac-9e52-91b37934e778 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:46:53.059621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d2e1a5f4-7f5b-46ac-9e52-91b37934e778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:46:53.074630 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1a5f4-7f5b-46ac-9e52-91b37934e778 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:46:53.080035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c51944-4c35-46c8-b62e-8c9880dce17b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:46:53.080679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c51944-4c35-46c8-b62e-8c9880dce17b service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:46:53.081015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c51944-4c35-46c8-b62e-8c9880dce17b service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:46:53.081440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c51944-4c35-46c8-b62e-8c9880dce17b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:46:53.201741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c51944-4c35-46c8-b62e-8c9880dce17b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:46:53.204172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c51944-4c35-46c8-b62e-8c9880dce17b service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:46:53.219220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1036/2053] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:46:53 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:02.878783 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=67,keystone:INSERT=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:03.210916 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=47 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:14.184154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4760af-9474-48eb-bddd-debbf48e00e3 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:14.248803 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4760af-9474-48eb-bddd-debbf48e00e3 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:14.253399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:14.254061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:14.254342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:14.254634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] PATH_INFO: `/v3/users/b4f7a8d8ffdd42a7a51be6ce15f1a982` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:14.256261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f7a8d8ffdd42a7a51be6ce15f1a982)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:14.258215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:14.306738 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f7a8d8ffdd42a7a51be6ce15f1a982'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:14.307389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f7a8d8ffdd42a7a51be6ce15f1a982'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:14.320268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f7a8d8ffdd42a7a51be6ce15f1a982'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:14.328031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f7a8d8ffdd42a7a51be6ce15f1a982'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:14.328816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f7a8d8ffdd42a7a51be6ce15f1a982'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:14.342614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1018/2054] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:14 2022] DELETE /identity/v3/users/b4f7a8d8ffdd42a7a51be6ce15f1a982 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:14.347292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43767e5-d22c-4a6d-9320-8090f1771f85 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:14.347758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43767e5-d22c-4a6d-9320-8090f1771f85 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:14.347849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43767e5-d22c-4a6d-9320-8090f1771f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:14.367192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d43767e5-d22c-4a6d-9320-8090f1771f85 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:14.368370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d43767e5-d22c-4a6d-9320-8090f1771f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:14.426354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1037/2055] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:14.436753 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:14.438178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:14.505329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18740f01-fe1a-4e6b-8627-191ea8616130 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:14.509821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c79f51-e71c-4e53-8bbe-46cb077903b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:14.510547 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c79f51-e71c-4e53-8bbe-46cb077903b7 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:14.510954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c79f51-e71c-4e53-8bbe-46cb077903b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:14.511336 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c79f51-e71c-4e53-8bbe-46cb077903b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:14.522263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c79f51-e71c-4e53-8bbe-46cb077903b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:14.523949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c79f51-e71c-4e53-8bbe-46cb077903b7 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:14.535077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1019/2056] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:14.896471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43767e5-d22c-4a6d-9320-8090f1771f85 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:14.898640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d43767e5-d22c-4a6d-9320-8090f1771f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:14.924096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43767e5-d22c-4a6d-9320-8090f1771f85 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:14.931023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:14.931664 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:14.932080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:14.932363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] PATH_INFO: `/v3/projects/722ff48d7acd47e8a7d696d0f5fa732f` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:14.934400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=722ff48d7acd47e8a7d696d0f5fa732f)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:14.936472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:14.990799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '722ff48d7acd47e8a7d696d0f5fa732f', 'request_id': 'req-9cd58392-60c0-49fd-92cd-aae666d1b2c8'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:14.998167 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '722ff48d7acd47e8a7d696d0f5fa732f', 'request_id': 'req-9cd58392-60c0-49fd-92cd-aae666d1b2c8'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.006108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '722ff48d7acd47e8a7d696d0f5fa732f', 'request_id': 'req-9cd58392-60c0-49fd-92cd-aae666d1b2c8'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.019325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] The token cache is being invalidate because project 722ff48d7acd47e8a7d696d0f5fa732f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:47:15.019620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.020264 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.022634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1038/2057] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:47:14 2022] DELETE /identity/v3/projects/722ff48d7acd47e8a7d696d0f5fa732f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:15.026587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c79f51-e71c-4e53-8bbe-46cb077903b7 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:15.036107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c79f51-e71c-4e53-8bbe-46cb077903b7 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:15.053867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:15.094381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12cd3d-08c7-43ea-a6f5-ffab0c7ecf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:15.095029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12cd3d-08c7-43ea-a6f5-ffab0c7ecf34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:15.095311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12cd3d-08c7-43ea-a6f5-ffab0c7ecf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:15.095580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12cd3d-08c7-43ea-a6f5-ffab0c7ecf34 admin admin] PATH_INFO: `/v3/users/1a78de9468ed4c80a8926eb45b6d6e78` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:15.096790 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12cd3d-08c7-43ea-a6f5-ffab0c7ecf34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a78de9468ed4c80a8926eb45b6d6e78)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:15.098807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12cd3d-08c7-43ea-a6f5-ffab0c7ecf34 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:15.142208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6a12cd3d-08c7-43ea-a6f5-ffab0c7ecf34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a78de9468ed4c80a8926eb45b6d6e78'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.142842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6a12cd3d-08c7-43ea-a6f5-ffab0c7ecf34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a78de9468ed4c80a8926eb45b6d6e78'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.154269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6a12cd3d-08c7-43ea-a6f5-ffab0c7ecf34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a78de9468ed4c80a8926eb45b6d6e78'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.160893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6a12cd3d-08c7-43ea-a6f5-ffab0c7ecf34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a78de9468ed4c80a8926eb45b6d6e78'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.162033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6a12cd3d-08c7-43ea-a6f5-ffab0c7ecf34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a78de9468ed4c80a8926eb45b6d6e78'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.174361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1020/2058] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:15 2022] DELETE /identity/v3/users/1a78de9468ed4c80a8926eb45b6d6e78 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:15.456469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:15.459406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:15.478434 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd58392-60c0-49fd-92cd-aae666d1b2c8 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:15.535294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ab814-42ba-447b-abf4-4f90153bb364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:15.535816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ab814-42ba-447b-abf4-4f90153bb364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:15.536093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ab814-42ba-447b-abf4-4f90153bb364 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:15.536975 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ab814-42ba-447b-abf4-4f90153bb364 admin admin] PATH_INFO: `/v3/projects/ef32c086b039433eb497f6a8e8ccfbd1` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:15.538144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ab814-42ba-447b-abf4-4f90153bb364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef32c086b039433eb497f6a8e8ccfbd1)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:15.540046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ab814-42ba-447b-abf4-4f90153bb364 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:15.569880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:15.587638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-436ab814-42ba-447b-abf4-4f90153bb364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef32c086b039433eb497f6a8e8ccfbd1', 'request_id': 'req-436ab814-42ba-447b-abf4-4f90153bb364'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.593826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-436ab814-42ba-447b-abf4-4f90153bb364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef32c086b039433eb497f6a8e8ccfbd1', 'request_id': 'req-436ab814-42ba-447b-abf4-4f90153bb364'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.599884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-436ab814-42ba-447b-abf4-4f90153bb364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef32c086b039433eb497f6a8e8ccfbd1', 'request_id': 'req-436ab814-42ba-447b-abf4-4f90153bb364'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.615984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-436ab814-42ba-447b-abf4-4f90153bb364 admin admin] The token cache is being invalidate because project ef32c086b039433eb497f6a8e8ccfbd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:47:15.616395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-436ab814-42ba-447b-abf4-4f90153bb364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.617049 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-436ab814-42ba-447b-abf4-4f90153bb364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:15.618789 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1039/2059] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:47:15 2022] DELETE /identity/v3/projects/ef32c086b039433eb497f6a8e8ccfbd1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:15.623440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8c5f6-1f60-4180-a936-e053997f7218 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:15.623919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8c5f6-1f60-4180-a936-e053997f7218 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:15.624015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8c5f6-1f60-4180-a936-e053997f7218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:15.644774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c2a8c5f6-1f60-4180-a936-e053997f7218 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:15.648493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c2a8c5f6-1f60-4180-a936-e053997f7218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:15.715114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1021/2060] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:15.723539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62c03d-33ef-4677-aa42-637bff89f717 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:15.723794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62c03d-33ef-4677-aa42-637bff89f717 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:15.724107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62c03d-33ef-4677-aa42-637bff89f717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:15.745466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-db62c03d-33ef-4677-aa42-637bff89f717 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:15.746730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-db62c03d-33ef-4677-aa42-637bff89f717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:15.755419 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1040/2061] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:15.759861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8c5f6-1f60-4180-a936-e053997f7218 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:15.770393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8c5f6-1f60-4180-a936-e053997f7218 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:15.775907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1a247-d63f-4def-a209-4e62a9a3a068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:15.776638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1a247-d63f-4def-a209-4e62a9a3a068 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:15.777003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1a247-d63f-4def-a209-4e62a9a3a068 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:15.777398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1a247-d63f-4def-a209-4e62a9a3a068 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:15.778179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1a247-d63f-4def-a209-4e62a9a3a068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:47:15.778558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1a247-d63f-4def-a209-4e62a9a3a068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:15.780263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1a247-d63f-4def-a209-4e62a9a3a068 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:15.796459 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1022/2062] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:47:15.797644 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62c03d-33ef-4677-aa42-637bff89f717 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:15.807250 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62c03d-33ef-4677-aa42-637bff89f717 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:15.812302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2dd42-87d5-4890-b862-4e4975dce9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:15.812979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2dd42-87d5-4890-b862-4e4975dce9a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:15.813339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2dd42-87d5-4890-b862-4e4975dce9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:15.813617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2dd42-87d5-4890-b862-4e4975dce9a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:15.816248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2dd42-87d5-4890-b862-4e4975dce9a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:15.818801 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2dd42-87d5-4890-b862-4e4975dce9a0 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:15.853082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1041/2063] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:15 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:15.857573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1a247-d63f-4def-a209-4e62a9a3a068 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:15.869644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1a247-d63f-4def-a209-4e62a9a3a068 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:15.902396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:15.934314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e616a-8c36-4c86-9304-7bd25895dfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:15.934932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e616a-8c36-4c86-9304-7bd25895dfb4 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:15.935212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e616a-8c36-4c86-9304-7bd25895dfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:15.935515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e616a-8c36-4c86-9304-7bd25895dfb4 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:15.936156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e616a-8c36-4c86-9304-7bd25895dfb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:15.938295 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e616a-8c36-4c86-9304-7bd25895dfb4 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:15.974458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1023/2064] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:15 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:15.978598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2dd42-87d5-4890-b862-4e4975dce9a0 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:15.991282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2dd42-87d5-4890-b862-4e4975dce9a0 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:16.000540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49081b95-77f7-44b2-b884-409503c38312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:16.001255 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49081b95-77f7-44b2-b884-409503c38312 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:16.001730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49081b95-77f7-44b2-b884-409503c38312 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:16.002227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49081b95-77f7-44b2-b884-409503c38312 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:16.002997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49081b95-77f7-44b2-b884-409503c38312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:16.017037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49081b95-77f7-44b2-b884-409503c38312 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:16.021449 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1042/2065] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:16.025890 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e616a-8c36-4c86-9304-7bd25895dfb4 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:16.046427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e616a-8c36-4c86-9304-7bd25895dfb4 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:16.051120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff96cc3-a52a-4661-a9ba-84cdca65acf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:16.051892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff96cc3-a52a-4661-a9ba-84cdca65acf0 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:16.052185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff96cc3-a52a-4661-a9ba-84cdca65acf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:16.052460 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff96cc3-a52a-4661-a9ba-84cdca65acf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:16.053240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff96cc3-a52a-4661-a9ba-84cdca65acf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:16.054869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff96cc3-a52a-4661-a9ba-84cdca65acf0 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:16.067182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1024/2066] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:16.071260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49081b95-77f7-44b2-b884-409503c38312 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:16.082672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49081b95-77f7-44b2-b884-409503c38312 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:16.088515 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd00426-abf6-4b7c-bdc1-6838b01956d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:16.089330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd00426-abf6-4b7c-bdc1-6838b01956d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:16.089479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd00426-abf6-4b7c-bdc1-6838b01956d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:16.092385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd00426-abf6-4b7c-bdc1-6838b01956d7 admin admin] PATH_INFO: `/v3/projects/3f68842ccdfa4382a3dda6b607fa90e2/users/9c261ef4393844d88d571a558eddcb40/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:16.110755 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd00426-abf6-4b7c-bdc1-6838b01956d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f68842ccdfa4382a3dda6b607fa90e2, user_id=9c261ef4393844d88d571a558eddcb40, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:16.113429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd00426-abf6-4b7c-bdc1-6838b01956d7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:16.130972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1043/2067] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:16 2022] PUT /identity/v3/projects/3f68842ccdfa4382a3dda6b607fa90e2/users/9c261ef4393844d88d571a558eddcb40/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:16.143786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f7307-581a-4053-8e34-a5dba3651917 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:16.144167 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f7307-581a-4053-8e34-a5dba3651917 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:16.144548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f7307-581a-4053-8e34-a5dba3651917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:16.174671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-de1f7307-581a-4053-8e34-a5dba3651917 None None] MFA Rules not processed for user `9c261ef4393844d88d571a558eddcb40`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:16.176179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-de1f7307-581a-4053-8e34-a5dba3651917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:16.257352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1025/2068] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:16.268508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd00426-abf6-4b7c-bdc1-6838b01956d7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:16.270286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2fd00426-abf6-4b7c-bdc1-6838b01956d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:16.326096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd00426-abf6-4b7c-bdc1-6838b01956d7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:16.329728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a042e36-6a65-4aae-b761-8ea41b512192 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:16.330334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a042e36-6a65-4aae-b761-8ea41b512192 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:16.330617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a042e36-6a65-4aae-b761-8ea41b512192 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:16.330894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a042e36-6a65-4aae-b761-8ea41b512192 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:16.339087 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a042e36-6a65-4aae-b761-8ea41b512192 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:16.340672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a042e36-6a65-4aae-b761-8ea41b512192 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:16.350322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1044/2069] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:20.508596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f7307-581a-4053-8e34-a5dba3651917 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:20.510096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-de1f7307-581a-4053-8e34-a5dba3651917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:20.577772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f7307-581a-4053-8e34-a5dba3651917 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:20.582365 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd8e82-9fcd-47a4-afaa-2e0f4ed6182a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:20.583030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd8e82-9fcd-47a4-afaa-2e0f4ed6182a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:20.583311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd8e82-9fcd-47a4-afaa-2e0f4ed6182a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:20.583584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd8e82-9fcd-47a4-afaa-2e0f4ed6182a admin admin] PATH_INFO: `/v3/users/980f097960df41528ac45d6c5a07c5f5` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:20.584716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd8e82-9fcd-47a4-afaa-2e0f4ed6182a admin admin] RBAC: Authorizing `identity:delete_user(user_id=980f097960df41528ac45d6c5a07c5f5)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:20.586888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd8e82-9fcd-47a4-afaa-2e0f4ed6182a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:20.628808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:20.640479 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9abd8e82-9fcd-47a4-afaa-2e0f4ed6182a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '980f097960df41528ac45d6c5a07c5f5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:20.641082 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9abd8e82-9fcd-47a4-afaa-2e0f4ed6182a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '980f097960df41528ac45d6c5a07c5f5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:20.657399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9abd8e82-9fcd-47a4-afaa-2e0f4ed6182a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '980f097960df41528ac45d6c5a07c5f5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:20.665448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9abd8e82-9fcd-47a4-afaa-2e0f4ed6182a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '980f097960df41528ac45d6c5a07c5f5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:20.666212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9abd8e82-9fcd-47a4-afaa-2e0f4ed6182a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '980f097960df41528ac45d6c5a07c5f5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:20.679493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1026/2070] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:20 2022] DELETE /identity/v3/users/980f097960df41528ac45d6c5a07c5f5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:20.859810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a042e36-6a65-4aae-b761-8ea41b512192 service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:20.886199 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a042e36-6a65-4aae-b761-8ea41b512192 service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:20.947970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:20.959249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b4507-7350-4785-ab6d-bf05baff3b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:20.960284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b4507-7350-4785-ab6d-bf05baff3b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:20.960869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b4507-7350-4785-ab6d-bf05baff3b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:20.961257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b4507-7350-4785-ab6d-bf05baff3b1b admin admin] PATH_INFO: `/v3/projects/7229cfcf52b64e83914f4ea2eb5627dd` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:20.962639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b4507-7350-4785-ab6d-bf05baff3b1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7229cfcf52b64e83914f4ea2eb5627dd)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:20.964676 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b4507-7350-4785-ab6d-bf05baff3b1b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:21.012241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-df8b4507-7350-4785-ab6d-bf05baff3b1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7229cfcf52b64e83914f4ea2eb5627dd', 'request_id': 'req-df8b4507-7350-4785-ab6d-bf05baff3b1b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:21.018464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-df8b4507-7350-4785-ab6d-bf05baff3b1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7229cfcf52b64e83914f4ea2eb5627dd', 'request_id': 'req-df8b4507-7350-4785-ab6d-bf05baff3b1b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:21.027772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-df8b4507-7350-4785-ab6d-bf05baff3b1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7229cfcf52b64e83914f4ea2eb5627dd', 'request_id': 'req-df8b4507-7350-4785-ab6d-bf05baff3b1b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:21.043217 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-df8b4507-7350-4785-ab6d-bf05baff3b1b admin admin] The token cache is being invalidate because project 7229cfcf52b64e83914f4ea2eb5627dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:47:21.043554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-df8b4507-7350-4785-ab6d-bf05baff3b1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:21.044224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-df8b4507-7350-4785-ab6d-bf05baff3b1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:21.046076 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1045/2071] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:47:20 2022] DELETE /identity/v3/projects/7229cfcf52b64e83914f4ea2eb5627dd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:21.051717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95236f-1f30-4439-b7bf-d0b579c11117 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.052058 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95236f-1f30-4439-b7bf-d0b579c11117 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.052341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95236f-1f30-4439-b7bf-d0b579c11117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.072785 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-2e95236f-1f30-4439-b7bf-d0b579c11117 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:21.074035 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2e95236f-1f30-4439-b7bf-d0b579c11117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:21.137287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1027/2072] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:21.145294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11ce90-e1f2-4352-ba01-455bc27b2411 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.146357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11ce90-e1f2-4352-ba01-455bc27b2411 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.146634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11ce90-e1f2-4352-ba01-455bc27b2411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.165950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-5a11ce90-e1f2-4352-ba01-455bc27b2411 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:21.167140 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-5a11ce90-e1f2-4352-ba01-455bc27b2411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:21.174782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1046/2073] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:21.179430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95236f-1f30-4439-b7bf-d0b579c11117 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:21.188883 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95236f-1f30-4439-b7bf-d0b579c11117 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:21.194126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444743e-7d85-487a-85e2-37eb6b2377fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:21.194735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444743e-7d85-487a-85e2-37eb6b2377fd admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.195014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444743e-7d85-487a-85e2-37eb6b2377fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.195285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444743e-7d85-487a-85e2-37eb6b2377fd admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.195923 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444743e-7d85-487a-85e2-37eb6b2377fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:47:21.196228 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444743e-7d85-487a-85e2-37eb6b2377fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:21.197850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444743e-7d85-487a-85e2-37eb6b2377fd admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:21.209137 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1028/2074] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:47:21.213273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11ce90-e1f2-4352-ba01-455bc27b2411 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:21.222701 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a11ce90-e1f2-4352-ba01-455bc27b2411 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:21.227368 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17761e3c-6e0b-4011-93e3-9000094f1ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:21.227984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17761e3c-6e0b-4011-93e3-9000094f1ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.228263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17761e3c-6e0b-4011-93e3-9000094f1ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.228537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17761e3c-6e0b-4011-93e3-9000094f1ec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.229162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17761e3c-6e0b-4011-93e3-9000094f1ec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:21.231131 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17761e3c-6e0b-4011-93e3-9000094f1ec8 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:21.259388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1047/2075] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:21 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:21.263534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444743e-7d85-487a-85e2-37eb6b2377fd admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:21.273868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444743e-7d85-487a-85e2-37eb6b2377fd admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:21.327590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa034ed-19c2-44a7-bb8c-f967758175ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:21.328183 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa034ed-19c2-44a7-bb8c-f967758175ac admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.328457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa034ed-19c2-44a7-bb8c-f967758175ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.328723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa034ed-19c2-44a7-bb8c-f967758175ac admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.329350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa034ed-19c2-44a7-bb8c-f967758175ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:21.331360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa034ed-19c2-44a7-bb8c-f967758175ac admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:21.352188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1029/2076] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:21 2022] POST /identity/v3/users => generated 294 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:21.356415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17761e3c-6e0b-4011-93e3-9000094f1ec8 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:21.368535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17761e3c-6e0b-4011-93e3-9000094f1ec8 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:21.373181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc009f8a-7925-4a9f-b3a6-39927e40fadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:21.373877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc009f8a-7925-4a9f-b3a6-39927e40fadd admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.374086 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc009f8a-7925-4a9f-b3a6-39927e40fadd admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.374365 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc009f8a-7925-4a9f-b3a6-39927e40fadd admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.375018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc009f8a-7925-4a9f-b3a6-39927e40fadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:21.376616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc009f8a-7925-4a9f-b3a6-39927e40fadd admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:21.394013 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1048/2077] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:21.397939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa034ed-19c2-44a7-bb8c-f967758175ac admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:21.408880 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa034ed-19c2-44a7-bb8c-f967758175ac admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:21.413771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5f8c3-4fdb-4388-81c8-4e8e5195b618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:21.414474 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5f8c3-4fdb-4388-81c8-4e8e5195b618 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.414751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5f8c3-4fdb-4388-81c8-4e8e5195b618 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.415025 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5f8c3-4fdb-4388-81c8-4e8e5195b618 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.415668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5f8c3-4fdb-4388-81c8-4e8e5195b618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:21.417255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5f8c3-4fdb-4388-81c8-4e8e5195b618 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:21.428532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1030/2078] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:21.433191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc009f8a-7925-4a9f-b3a6-39927e40fadd admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:21.443357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc009f8a-7925-4a9f-b3a6-39927e40fadd admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:21.447989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11f34d-f6a7-43bb-b874-b32f2ee6e3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:21.448668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11f34d-f6a7-43bb-b874-b32f2ee6e3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.448961 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11f34d-f6a7-43bb-b874-b32f2ee6e3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.449228 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11f34d-f6a7-43bb-b874-b32f2ee6e3c3 admin admin] PATH_INFO: `/v3/projects/d5dd4bb30cd549ee85668cffe4563815/users/f04c7ade3fb6475bba9522a2a7081a21/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.465459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11f34d-f6a7-43bb-b874-b32f2ee6e3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5dd4bb30cd549ee85668cffe4563815, user_id=f04c7ade3fb6475bba9522a2a7081a21, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:21.468153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11f34d-f6a7-43bb-b874-b32f2ee6e3c3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:21.482105 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1049/2079] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:21 2022] PUT /identity/v3/projects/d5dd4bb30cd549ee85668cffe4563815/users/f04c7ade3fb6475bba9522a2a7081a21/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:21.493924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b45a2-3936-408d-9b02-22aeafbed866 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.494212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b45a2-3936-408d-9b02-22aeafbed866 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.494498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b45a2-3936-408d-9b02-22aeafbed866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.523364 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-265b45a2-3936-408d-9b02-22aeafbed866 None None] MFA Rules not processed for user `f04c7ade3fb6475bba9522a2a7081a21`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:21.524958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-265b45a2-3936-408d-9b02-22aeafbed866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:21.535915 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:21.591938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1031/2080] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:21.597111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11f34d-f6a7-43bb-b874-b32f2ee6e3c3 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:21.703616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11f34d-f6a7-43bb-b874-b32f2ee6e3c3 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:21.710549 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf657cb7-071e-4430-baf0-e30cc07d0ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:21.711151 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf657cb7-071e-4430-baf0-e30cc07d0ebd admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.711432 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf657cb7-071e-4430-baf0-e30cc07d0ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.711704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf657cb7-071e-4430-baf0-e30cc07d0ebd admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.712353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf657cb7-071e-4430-baf0-e30cc07d0ebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:21.714378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf657cb7-071e-4430-baf0-e30cc07d0ebd admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:21.751367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1050/2081] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:21 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:21.753049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b45a2-3936-408d-9b02-22aeafbed866 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:21.764566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b45a2-3936-408d-9b02-22aeafbed866 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:21.839929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44595f31-b206-45cb-9159-20e6aec5f34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:21.840557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44595f31-b206-45cb-9159-20e6aec5f34f admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.840842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44595f31-b206-45cb-9159-20e6aec5f34f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.841118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44595f31-b206-45cb-9159-20e6aec5f34f admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.841794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44595f31-b206-45cb-9159-20e6aec5f34f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:21.843811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44595f31-b206-45cb-9159-20e6aec5f34f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:21.870418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1032/2082] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:21 2022] POST /identity/v3/users => generated 300 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:21.871603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf657cb7-071e-4430-baf0-e30cc07d0ebd admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:21.878572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:21.884204 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf657cb7-071e-4430-baf0-e30cc07d0ebd admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:21.891154 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abe930-187e-492c-b6cc-d1218e0737ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:21.891918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abe930-187e-492c-b6cc-d1218e0737ce admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.892546 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abe930-187e-492c-b6cc-d1218e0737ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.893155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abe930-187e-492c-b6cc-d1218e0737ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.894332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abe930-187e-492c-b6cc-d1218e0737ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:21.896100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abe930-187e-492c-b6cc-d1218e0737ce admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:21.908423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1051/2083] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:47:21.914688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44595f31-b206-45cb-9159-20e6aec5f34f admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:21.939184 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44595f31-b206-45cb-9159-20e6aec5f34f admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:21.944779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43197b58-33c1-43e9-9859-035849301bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:21.945640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43197b58-33c1-43e9-9859-035849301bcb admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:21.946341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43197b58-33c1-43e9-9859-035849301bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:21.946614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43197b58-33c1-43e9-9859-035849301bcb admin admin] PATH_INFO: `/v3/projects/d7202c9592d34d319335157adc886585/users/b5091a1df3de4d9b9ac50019a103009b/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:21.965821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43197b58-33c1-43e9-9859-035849301bcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7202c9592d34d319335157adc886585, user_id=b5091a1df3de4d9b9ac50019a103009b, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:21.968602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43197b58-33c1-43e9-9859-035849301bcb admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:21.990202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1033/2084] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:21 2022] PUT /identity/v3/projects/d7202c9592d34d319335157adc886585/users/b5091a1df3de4d9b9ac50019a103009b/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:21.991456 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abe930-187e-492c-b6cc-d1218e0737ce admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:22.003714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abe930-187e-492c-b6cc-d1218e0737ce admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:22.065631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e2e61-3ccc-4602-989f-780758b483c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:22.066263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2e61-3ccc-4602-989f-780758b483c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:22.066537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2e61-3ccc-4602-989f-780758b483c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:22.066806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2e61-3ccc-4602-989f-780758b483c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:22.067569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e2e61-3ccc-4602-989f-780758b483c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:22.069157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e2e61-3ccc-4602-989f-780758b483c1 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:22.078940 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1052/2085] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:22.083573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43197b58-33c1-43e9-9859-035849301bcb admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:22.096768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43197b58-33c1-43e9-9859-035849301bcb admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:22.103154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf21ae-c15d-4165-9756-55eb0355c46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:22.104191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf21ae-c15d-4165-9756-55eb0355c46c admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:22.104808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf21ae-c15d-4165-9756-55eb0355c46c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:22.105007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf21ae-c15d-4165-9756-55eb0355c46c admin admin] PATH_INFO: `/v3/projects/d7202c9592d34d319335157adc886585/users/b5091a1df3de4d9b9ac50019a103009b/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:22.106837 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf21ae-c15d-4165-9756-55eb0355c46c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7202c9592d34d319335157adc886585, user_id=b5091a1df3de4d9b9ac50019a103009b, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:22.109483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf21ae-c15d-4165-9756-55eb0355c46c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:22.122316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1034/2086] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:22 2022] PUT /identity/v3/projects/d7202c9592d34d319335157adc886585/users/b5091a1df3de4d9b9ac50019a103009b/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:22.134866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b45aa7-cb45-4414-a8cd-70057bdc76db None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:22.135338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b45aa7-cb45-4414-a8cd-70057bdc76db None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:22.135730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b45aa7-cb45-4414-a8cd-70057bdc76db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:22.164495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b0b45aa7-cb45-4414-a8cd-70057bdc76db None None] MFA Rules not processed for user `b5091a1df3de4d9b9ac50019a103009b`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:22.165826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b0b45aa7-cb45-4414-a8cd-70057bdc76db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:22.233906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1053/2087] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:22.245114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf21ae-c15d-4165-9756-55eb0355c46c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:22.246891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-99bf21ae-c15d-4165-9756-55eb0355c46c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:22.304712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf21ae-c15d-4165-9756-55eb0355c46c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:22.308520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2dfae-260d-45b5-9628-10fa13861e2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:22.309144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2dfae-260d-45b5-9628-10fa13861e2c service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:22.309453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2dfae-260d-45b5-9628-10fa13861e2c service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:22.309771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2dfae-260d-45b5-9628-10fa13861e2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:22.320054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2dfae-260d-45b5-9628-10fa13861e2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:22.322368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2dfae-260d-45b5-9628-10fa13861e2c service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:22.333458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1035/2088] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:47:22.852990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b45aa7-cb45-4414-a8cd-70057bdc76db None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:22.914090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b45aa7-cb45-4414-a8cd-70057bdc76db None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:22.918615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:22.919352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:22.919552 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:22.919826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] PATH_INFO: `/v3/users/f04c7ade3fb6475bba9522a2a7081a21` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:22.920948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f04c7ade3fb6475bba9522a2a7081a21)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:22.922905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:22.939587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:22.972318 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f04c7ade3fb6475bba9522a2a7081a21'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:22.972974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f04c7ade3fb6475bba9522a2a7081a21'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:22.980007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f04c7ade3fb6475bba9522a2a7081a21'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:22.980655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f04c7ade3fb6475bba9522a2a7081a21'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:22.993207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f04c7ade3fb6475bba9522a2a7081a21'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:23.006895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1054/2089] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:22 2022] DELETE /identity/v3/users/f04c7ade3fb6475bba9522a2a7081a21 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:23.011388 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c00e0-f497-4e16-9f5c-8e4229879c7f None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:23.011855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c00e0-f497-4e16-9f5c-8e4229879c7f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:23.012122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c00e0-f497-4e16-9f5c-8e4229879c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:23.026275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:23.032629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-7d5c00e0-f497-4e16-9f5c-8e4229879c7f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:23.034412 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7d5c00e0-f497-4e16-9f5c-8e4229879c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:23.100348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1036/2090] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:23.109656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:23.111130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:23.168384 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b032aa-f665-44df-8def-34b49b105a2e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:23.171993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892e962-0db4-4b68-bdc7-cbe4fc6b0881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:23.172593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892e962-0db4-4b68-bdc7-cbe4fc6b0881 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:23.172872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892e962-0db4-4b68-bdc7-cbe4fc6b0881 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:23.173142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892e962-0db4-4b68-bdc7-cbe4fc6b0881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:23.182752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892e962-0db4-4b68-bdc7-cbe4fc6b0881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:23.184170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892e962-0db4-4b68-bdc7-cbe4fc6b0881 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:23.193461 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1055/2091] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:23.458500 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c00e0-f497-4e16-9f5c-8e4229879c7f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:23.460037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7d5c00e0-f497-4e16-9f5c-8e4229879c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:23.472868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c00e0-f497-4e16-9f5c-8e4229879c7f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:23.479291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:23.480151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:23.480437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:23.480730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] PATH_INFO: `/v3/projects/d5dd4bb30cd549ee85668cffe4563815` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:23.481886 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5dd4bb30cd549ee85668cffe4563815)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:23.483806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:23.533105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5dd4bb30cd549ee85668cffe4563815', 'request_id': 'req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:23.539292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5dd4bb30cd549ee85668cffe4563815', 'request_id': 'req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:23.544782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5dd4bb30cd549ee85668cffe4563815', 'request_id': 'req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:23.557292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] The token cache is being invalidate because project d5dd4bb30cd549ee85668cffe4563815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:47:23.557609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:23.558271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:23.559755 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1037/2092] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:47:23 2022] DELETE /identity/v3/projects/d5dd4bb30cd549ee85668cffe4563815 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:23.563830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892e962-0db4-4b68-bdc7-cbe4fc6b0881 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:23.573531 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892e962-0db4-4b68-bdc7-cbe4fc6b0881 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:23.630008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff2950-761d-4e04-80c5-2b39295e8379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:23.630634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff2950-761d-4e04-80c5-2b39295e8379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:23.630914 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff2950-761d-4e04-80c5-2b39295e8379 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:23.631188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff2950-761d-4e04-80c5-2b39295e8379 admin admin] PATH_INFO: `/v3/users/b5091a1df3de4d9b9ac50019a103009b` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:23.632326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff2950-761d-4e04-80c5-2b39295e8379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5091a1df3de4d9b9ac50019a103009b)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:23.634422 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff2950-761d-4e04-80c5-2b39295e8379 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:23.674145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:23.680917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d4ff2950-761d-4e04-80c5-2b39295e8379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5091a1df3de4d9b9ac50019a103009b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:23.681906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d4ff2950-761d-4e04-80c5-2b39295e8379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5091a1df3de4d9b9ac50019a103009b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:23.689175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d4ff2950-761d-4e04-80c5-2b39295e8379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5091a1df3de4d9b9ac50019a103009b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:23.689834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d4ff2950-761d-4e04-80c5-2b39295e8379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5091a1df3de4d9b9ac50019a103009b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:23.702203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d4ff2950-761d-4e04-80c5-2b39295e8379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5091a1df3de4d9b9ac50019a103009b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:23.715034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1056/2093] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:23 2022] DELETE /identity/v3/users/b5091a1df3de4d9b9ac50019a103009b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:23.983596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:23.985197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:23.998689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a7fcf-17b6-4c1a-b33d-8c3be8654b57 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:24.054091 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961ba33-bf16-4696-bb00-a3d3d637d995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:24.054718 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961ba33-bf16-4696-bb00-a3d3d637d995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:24.054994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961ba33-bf16-4696-bb00-a3d3d637d995 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:24.055262 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961ba33-bf16-4696-bb00-a3d3d637d995 admin admin] PATH_INFO: `/v3/projects/d7202c9592d34d319335157adc886585` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:24.056362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961ba33-bf16-4696-bb00-a3d3d637d995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7202c9592d34d319335157adc886585)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:24.058298 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961ba33-bf16-4696-bb00-a3d3d637d995 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:24.093216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:24.107156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5961ba33-bf16-4696-bb00-a3d3d637d995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7202c9592d34d319335157adc886585', 'request_id': 'req-5961ba33-bf16-4696-bb00-a3d3d637d995'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:24.116835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5961ba33-bf16-4696-bb00-a3d3d637d995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7202c9592d34d319335157adc886585', 'request_id': 'req-5961ba33-bf16-4696-bb00-a3d3d637d995'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:24.128854 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5961ba33-bf16-4696-bb00-a3d3d637d995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7202c9592d34d319335157adc886585', 'request_id': 'req-5961ba33-bf16-4696-bb00-a3d3d637d995'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:24.145721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5961ba33-bf16-4696-bb00-a3d3d637d995 admin admin] The token cache is being invalidate because project d7202c9592d34d319335157adc886585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:47:24.146029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5961ba33-bf16-4696-bb00-a3d3d637d995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:24.146699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5961ba33-bf16-4696-bb00-a3d3d637d995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:24.148431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1038/2094] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:47:23 2022] DELETE /identity/v3/projects/d7202c9592d34d319335157adc886585 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:24.153247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ffc45-50a6-4da1-9992-b6619b6b27ac None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:24.153741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ffc45-50a6-4da1-9992-b6619b6b27ac None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:24.153959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ffc45-50a6-4da1-9992-b6619b6b27ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:24.180872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-278ffc45-50a6-4da1-9992-b6619b6b27ac None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:24.182112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-278ffc45-50a6-4da1-9992-b6619b6b27ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:24.251995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1057/2095] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:24.258236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5fea4-4121-42a4-a0b9-67dccdf4c788 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:24.258615 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5fea4-4121-42a4-a0b9-67dccdf4c788 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:24.258958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5fea4-4121-42a4-a0b9-67dccdf4c788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:24.278886 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-add5fea4-4121-42a4-a0b9-67dccdf4c788 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:24.280141 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-add5fea4-4121-42a4-a0b9-67dccdf4c788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:24.288714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1039/2096] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:24.293017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ffc45-50a6-4da1-9992-b6619b6b27ac None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:24.303084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ffc45-50a6-4da1-9992-b6619b6b27ac None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:24.307517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208f302-0c72-4c2f-bbea-e0dce19ad54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:24.308102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208f302-0c72-4c2f-bbea-e0dce19ad54f admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:24.308378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208f302-0c72-4c2f-bbea-e0dce19ad54f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:24.308645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208f302-0c72-4c2f-bbea-e0dce19ad54f admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:24.309306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208f302-0c72-4c2f-bbea-e0dce19ad54f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:47:24.309601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208f302-0c72-4c2f-bbea-e0dce19ad54f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:24.311238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208f302-0c72-4c2f-bbea-e0dce19ad54f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:24.323527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1058/2097] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:47:24.327748 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5fea4-4121-42a4-a0b9-67dccdf4c788 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:24.336803 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5fea4-4121-42a4-a0b9-67dccdf4c788 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:24.341363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a11c0f-1945-4f4f-9d80-ba60163beab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:24.341988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a11c0f-1945-4f4f-9d80-ba60163beab6 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:24.342265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a11c0f-1945-4f4f-9d80-ba60163beab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:24.342535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a11c0f-1945-4f4f-9d80-ba60163beab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:24.343168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a11c0f-1945-4f4f-9d80-ba60163beab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:24.345321 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a11c0f-1945-4f4f-9d80-ba60163beab6 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:24.371258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1040/2098] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:24 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:47:24.375320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208f302-0c72-4c2f-bbea-e0dce19ad54f admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:24.385665 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208f302-0c72-4c2f-bbea-e0dce19ad54f admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:24.440091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c8f24-ef8e-4854-8086-c1b7a831c541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:24.440697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c8f24-ef8e-4854-8086-c1b7a831c541 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:24.440980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c8f24-ef8e-4854-8086-c1b7a831c541 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:24.441254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c8f24-ef8e-4854-8086-c1b7a831c541 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:24.441927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c8f24-ef8e-4854-8086-c1b7a831c541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:24.444445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c8f24-ef8e-4854-8086-c1b7a831c541 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:24.468393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1059/2099] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:24 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:24.469867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a11c0f-1945-4f4f-9d80-ba60163beab6 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:24.479942 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a11c0f-1945-4f4f-9d80-ba60163beab6 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:24.484482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882300b-1539-48ec-bea6-7e0831070535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:24.485098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882300b-1539-48ec-bea6-7e0831070535 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:24.485379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882300b-1539-48ec-bea6-7e0831070535 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:24.485653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882300b-1539-48ec-bea6-7e0831070535 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:24.486322 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882300b-1539-48ec-bea6-7e0831070535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:24.487901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882300b-1539-48ec-bea6-7e0831070535 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:24.498058 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1041/2100] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:24.501806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c8f24-ef8e-4854-8086-c1b7a831c541 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:24.512181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c8f24-ef8e-4854-8086-c1b7a831c541 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:24.516895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad7084-e0cd-4812-af2b-5ee56117ec8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:24.517493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad7084-e0cd-4812-af2b-5ee56117ec8d admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:24.517809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad7084-e0cd-4812-af2b-5ee56117ec8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:24.518086 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad7084-e0cd-4812-af2b-5ee56117ec8d admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:24.518723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad7084-e0cd-4812-af2b-5ee56117ec8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:24.520935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad7084-e0cd-4812-af2b-5ee56117ec8d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:24.531678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1060/2101] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:24.536271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882300b-1539-48ec-bea6-7e0831070535 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:24.546013 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882300b-1539-48ec-bea6-7e0831070535 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:24.550559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595c935-91ed-4002-81a1-4f51414a89db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:24.551190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595c935-91ed-4002-81a1-4f51414a89db admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:24.551511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595c935-91ed-4002-81a1-4f51414a89db admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:24.551785 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595c935-91ed-4002-81a1-4f51414a89db admin admin] PATH_INFO: `/v3/projects/738514e187e44365980b7794bc9923f2/users/e091246128db4727ba507029b25d3774/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:24.565951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595c935-91ed-4002-81a1-4f51414a89db admin admin] RBAC: Authorizing `identity:create_grant(project_id=738514e187e44365980b7794bc9923f2, user_id=e091246128db4727ba507029b25d3774, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:24.569150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595c935-91ed-4002-81a1-4f51414a89db admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:24.581827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1042/2102] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:24 2022] PUT /identity/v3/projects/738514e187e44365980b7794bc9923f2/users/e091246128db4727ba507029b25d3774/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:24.587424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cc38d-85b1-4bdb-b6d7-4dd093eb9d4e None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:24.587733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cc38d-85b1-4bdb-b6d7-4dd093eb9d4e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:24.588003 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cc38d-85b1-4bdb-b6d7-4dd093eb9d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:24.606550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-621cc38d-85b1-4bdb-b6d7-4dd093eb9d4e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:24.607733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-621cc38d-85b1-4bdb-b6d7-4dd093eb9d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:24.623593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:24.667940 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1061/2103] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:24.677416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595c935-91ed-4002-81a1-4f51414a89db admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:24.679125 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d595c935-91ed-4002-81a1-4f51414a89db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:24.735594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595c935-91ed-4002-81a1-4f51414a89db admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:24.739231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd47c0d-4291-4100-9a78-f3df66ab800f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:24.739833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd47c0d-4291-4100-9a78-f3df66ab800f service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:24.740111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd47c0d-4291-4100-9a78-f3df66ab800f service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:24.740380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd47c0d-4291-4100-9a78-f3df66ab800f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:24.749060 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd47c0d-4291-4100-9a78-f3df66ab800f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:24.750517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd47c0d-4291-4100-9a78-f3df66ab800f service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:24.760109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1043/2104] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:47:25.435668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc6aee-1f80-4d2c-817d-7f3828581556 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:25.436188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc6aee-1f80-4d2c-817d-7f3828581556 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:25.436278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc6aee-1f80-4d2c-817d-7f3828581556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:25.470147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-25cc6aee-1f80-4d2c-817d-7f3828581556 None None] MFA Rules not processed for user `e091246128db4727ba507029b25d3774`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:25.471483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-25cc6aee-1f80-4d2c-817d-7f3828581556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:25.542031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1062/2105] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:25.546820 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd47c0d-4291-4100-9a78-f3df66ab800f service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:25.557219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd47c0d-4291-4100-9a78-f3df66ab800f service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:25.561849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135c3f5-2af4-4e22-8242-cb0852ef6be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:25.562450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135c3f5-2af4-4e22-8242-cb0852ef6be2 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:25.562733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135c3f5-2af4-4e22-8242-cb0852ef6be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:25.563008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135c3f5-2af4-4e22-8242-cb0852ef6be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:25.563654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135c3f5-2af4-4e22-8242-cb0852ef6be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:25.565559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135c3f5-2af4-4e22-8242-cb0852ef6be2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:25.595694 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1044/2106] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:25 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:25.600118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc6aee-1f80-4d2c-817d-7f3828581556 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:25.611194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc6aee-1f80-4d2c-817d-7f3828581556 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:25.666861 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241ae84-d31b-43e7-83dc-5064368de961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:25.667471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241ae84-d31b-43e7-83dc-5064368de961 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:25.667777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241ae84-d31b-43e7-83dc-5064368de961 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:25.668051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241ae84-d31b-43e7-83dc-5064368de961 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:25.668693 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241ae84-d31b-43e7-83dc-5064368de961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:25.670658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241ae84-d31b-43e7-83dc-5064368de961 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:25.691778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1063/2107] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:25 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:25.695740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135c3f5-2af4-4e22-8242-cb0852ef6be2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:25.705995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135c3f5-2af4-4e22-8242-cb0852ef6be2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:25.711064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b1240-d665-4628-9a87-2b7046e1d091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:25.711657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b1240-d665-4628-9a87-2b7046e1d091 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:25.711936 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b1240-d665-4628-9a87-2b7046e1d091 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:25.712209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b1240-d665-4628-9a87-2b7046e1d091 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:25.712878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b1240-d665-4628-9a87-2b7046e1d091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:25.714521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b1240-d665-4628-9a87-2b7046e1d091 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:25.725730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1045/2108] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:25.729662 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241ae84-d31b-43e7-83dc-5064368de961 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:25.739520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241ae84-d31b-43e7-83dc-5064368de961 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:25.744117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc99d-874c-45e3-8130-376176db8f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:25.744757 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc99d-874c-45e3-8130-376176db8f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:25.745040 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc99d-874c-45e3-8130-376176db8f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:25.745337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dc99d-874c-45e3-8130-376176db8f93 admin admin] PATH_INFO: `/v3/projects/e94026aaac6a4201aace07d8dec67117/users/2e3a76b0ef1744bbb1b877ef0b40b80b/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:25.759703 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dc99d-874c-45e3-8130-376176db8f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e94026aaac6a4201aace07d8dec67117, user_id=2e3a76b0ef1744bbb1b877ef0b40b80b, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:25.762416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550dc99d-874c-45e3-8130-376176db8f93 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:25.775565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1064/2109] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:25 2022] PUT /identity/v3/projects/e94026aaac6a4201aace07d8dec67117/users/2e3a76b0ef1744bbb1b877ef0b40b80b/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:25.779165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b1240-d665-4628-9a87-2b7046e1d091 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:25.789586 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b1240-d665-4628-9a87-2b7046e1d091 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:25.813145 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:25.846526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5210e-4590-483a-bbb6-da94aa4b27c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:25.847126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5210e-4590-483a-bbb6-da94aa4b27c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:25.847403 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5210e-4590-483a-bbb6-da94aa4b27c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:25.847674 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5210e-4590-483a-bbb6-da94aa4b27c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:25.848366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5210e-4590-483a-bbb6-da94aa4b27c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:25.849985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5210e-4590-483a-bbb6-da94aa4b27c1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:25.859353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1046/2110] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:25.863543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc99d-874c-45e3-8130-376176db8f93 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:25.874600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dc99d-874c-45e3-8130-376176db8f93 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:25.879099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc142fe-4d33-4146-aa72-ebdddcb5a9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:25.879831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc142fe-4d33-4146-aa72-ebdddcb5a9da admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:25.880121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc142fe-4d33-4146-aa72-ebdddcb5a9da admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:25.880393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc142fe-4d33-4146-aa72-ebdddcb5a9da admin admin] PATH_INFO: `/v3/projects/e94026aaac6a4201aace07d8dec67117/users/2e3a76b0ef1744bbb1b877ef0b40b80b/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:25.882301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc142fe-4d33-4146-aa72-ebdddcb5a9da admin admin] RBAC: Authorizing `identity:create_grant(project_id=e94026aaac6a4201aace07d8dec67117, user_id=2e3a76b0ef1744bbb1b877ef0b40b80b, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:25.884956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc142fe-4d33-4146-aa72-ebdddcb5a9da admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:25.895761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1065/2111] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:25 2022] PUT /identity/v3/projects/e94026aaac6a4201aace07d8dec67117/users/2e3a76b0ef1744bbb1b877ef0b40b80b/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:26.727329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e2046-83b6-427b-b554-7308944a9a8d None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:26.727819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e2046-83b6-427b-b554-7308944a9a8d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:26.727949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e2046-83b6-427b-b554-7308944a9a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:26.757501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-fe9e2046-83b6-427b-b554-7308944a9a8d None None] MFA Rules not processed for user `2e3a76b0ef1744bbb1b877ef0b40b80b`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:26.758797 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-fe9e2046-83b6-427b-b554-7308944a9a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:26.829397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1047/2112] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:26.840549 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc142fe-4d33-4146-aa72-ebdddcb5a9da admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:26.842244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9fc142fe-4d33-4146-aa72-ebdddcb5a9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:26.849633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:26.905446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc142fe-4d33-4146-aa72-ebdddcb5a9da admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:26.909373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994256f-e3b5-432d-a3bb-cbc5313f9b9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:26.910016 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994256f-e3b5-432d-a3bb-cbc5313f9b9f service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:26.910420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994256f-e3b5-432d-a3bb-cbc5313f9b9f service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:26.911241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994256f-e3b5-432d-a3bb-cbc5313f9b9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:26.967687 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994256f-e3b5-432d-a3bb-cbc5313f9b9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:26.969398 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994256f-e3b5-432d-a3bb-cbc5313f9b9f service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:26.978178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1066/2113] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:29.412584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e2046-83b6-427b-b554-7308944a9a8d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:29.423406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e2046-83b6-427b-b554-7308944a9a8d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:29.427808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9a6c4-a2a0-4924-a0bc-72dcf2126dc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:29.428519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9a6c4-a2a0-4924-a0bc-72dcf2126dc2 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:29.428860 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9a6c4-a2a0-4924-a0bc-72dcf2126dc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:29.429221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9a6c4-a2a0-4924-a0bc-72dcf2126dc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:29.438882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9a6c4-a2a0-4924-a0bc-72dcf2126dc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:29.440334 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9a6c4-a2a0-4924-a0bc-72dcf2126dc2 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:29.449805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1048/2114] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:36.974172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=46 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:39.445612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=57 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:45.400738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994256f-e3b5-432d-a3bb-cbc5313f9b9f service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:45.402168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7994256f-e3b5-432d-a3bb-cbc5313f9b9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:45.468332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994256f-e3b5-432d-a3bb-cbc5313f9b9f service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:45.473040 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:45.473668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:45.473985 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:45.474265 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] PATH_INFO: `/v3/users/4eb0674c8b274fa29908ac76842e7a4d` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:45.475850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eb0674c8b274fa29908ac76842e7a4d)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:45.478598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:45.525081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eb0674c8b274fa29908ac76842e7a4d'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:45.525767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eb0674c8b274fa29908ac76842e7a4d'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:45.532642 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eb0674c8b274fa29908ac76842e7a4d'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:45.533273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eb0674c8b274fa29908ac76842e7a4d'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:45.544833 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eb0674c8b274fa29908ac76842e7a4d'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:45.556796 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1067/2115] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:45 2022] DELETE /identity/v3/users/4eb0674c8b274fa29908ac76842e7a4d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:45.561220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b5452-06e2-421b-9810-7bf635d8b02a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:45.561709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b5452-06e2-421b-9810-7bf635d8b02a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:45.561819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b5452-06e2-421b-9810-7bf635d8b02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:45.580587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-075b5452-06e2-421b-9810-7bf635d8b02a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:45.582423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-075b5452-06e2-421b-9810-7bf635d8b02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:45.640436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1049/2116] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:45.650120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:45.651539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:45.711502 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb31ff-d2f7-4ef7-8697-e4323e19f10e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:45.715234 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7640a2-290d-4250-b0d1-49f33d3e1834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:45.715841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7640a2-290d-4250-b0d1-49f33d3e1834 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:45.716124 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7640a2-290d-4250-b0d1-49f33d3e1834 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:45.716433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7640a2-290d-4250-b0d1-49f33d3e1834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:45.726119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7640a2-290d-4250-b0d1-49f33d3e1834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:45.727575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7640a2-290d-4250-b0d1-49f33d3e1834 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:45.737220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1068/2117] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:46.036333 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b5452-06e2-421b-9810-7bf635d8b02a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:46.038504 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-075b5452-06e2-421b-9810-7bf635d8b02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:46.051440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b5452-06e2-421b-9810-7bf635d8b02a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:46.057943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47391989-a54a-4e81-acac-f45599cc9760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:46.058665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47391989-a54a-4e81-acac-f45599cc9760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.059016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47391989-a54a-4e81-acac-f45599cc9760 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.059370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47391989-a54a-4e81-acac-f45599cc9760 admin admin] PATH_INFO: `/v3/projects/805addb87ebc4c08ab117be25b17b6de` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.062000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47391989-a54a-4e81-acac-f45599cc9760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=805addb87ebc4c08ab117be25b17b6de)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:46.064252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47391989-a54a-4e81-acac-f45599cc9760 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:46.117362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-47391989-a54a-4e81-acac-f45599cc9760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '805addb87ebc4c08ab117be25b17b6de', 'request_id': 'req-47391989-a54a-4e81-acac-f45599cc9760'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:46.124185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-47391989-a54a-4e81-acac-f45599cc9760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '805addb87ebc4c08ab117be25b17b6de', 'request_id': 'req-47391989-a54a-4e81-acac-f45599cc9760'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:46.131262 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-47391989-a54a-4e81-acac-f45599cc9760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '805addb87ebc4c08ab117be25b17b6de', 'request_id': 'req-47391989-a54a-4e81-acac-f45599cc9760'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:46.144859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-47391989-a54a-4e81-acac-f45599cc9760 admin admin] The token cache is being invalidate because project 805addb87ebc4c08ab117be25b17b6de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:47:46.145173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-47391989-a54a-4e81-acac-f45599cc9760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:46.145855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-47391989-a54a-4e81-acac-f45599cc9760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:46.147337 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1050/2118] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:47:46 2022] DELETE /identity/v3/projects/805addb87ebc4c08ab117be25b17b6de => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:46.152216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645f58-1b28-47ea-8300-360ddfaaddf6 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.152507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645f58-1b28-47ea-8300-360ddfaaddf6 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.152779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645f58-1b28-47ea-8300-360ddfaaddf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.172501 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-c4645f58-1b28-47ea-8300-360ddfaaddf6 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:46.173741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c4645f58-1b28-47ea-8300-360ddfaaddf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:46.192263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:46.239225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1069/2119] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:46.247077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24fe9-8f24-4595-9a28-ee0fc6f63246 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.247360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24fe9-8f24-4595-9a28-ee0fc6f63246 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.247634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d24fe9-8f24-4595-9a28-ee0fc6f63246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.268328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f9d24fe9-8f24-4595-9a28-ee0fc6f63246 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:46.269564 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f9d24fe9-8f24-4595-9a28-ee0fc6f63246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:46.277993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1051/2120] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:46.282213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645f58-1b28-47ea-8300-360ddfaaddf6 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:46.292215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645f58-1b28-47ea-8300-360ddfaaddf6 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:46.296714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e111f-088d-40d2-97c7-f415c2cf1f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:46.297359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e111f-088d-40d2-97c7-f415c2cf1f35 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.297644 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e111f-088d-40d2-97c7-f415c2cf1f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.297949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e111f-088d-40d2-97c7-f415c2cf1f35 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.298593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e111f-088d-40d2-97c7-f415c2cf1f35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:47:46.298895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e111f-088d-40d2-97c7-f415c2cf1f35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:46.301194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e111f-088d-40d2-97c7-f415c2cf1f35 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:46.312795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1070/2121] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:47:46.317370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d24fe9-8f24-4595-9a28-ee0fc6f63246 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:46.328278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d24fe9-8f24-4595-9a28-ee0fc6f63246 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:46.333644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df924459-c323-4119-8760-85e88b0e197b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:46.334275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df924459-c323-4119-8760-85e88b0e197b admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.334566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df924459-c323-4119-8760-85e88b0e197b admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.334834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df924459-c323-4119-8760-85e88b0e197b admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.335508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df924459-c323-4119-8760-85e88b0e197b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:46.337443 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df924459-c323-4119-8760-85e88b0e197b admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:46.371002 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1052/2122] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:46 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:46.373582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e111f-088d-40d2-97c7-f415c2cf1f35 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:46.383551 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e111f-088d-40d2-97c7-f415c2cf1f35 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:46.444992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9714e-a0bb-44b0-9c78-b4b960c5369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:46.446158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9714e-a0bb-44b0-9c78-b4b960c5369f admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.446479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9714e-a0bb-44b0-9c78-b4b960c5369f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.446864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9714e-a0bb-44b0-9c78-b4b960c5369f admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.448090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9714e-a0bb-44b0-9c78-b4b960c5369f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:46.450950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9714e-a0bb-44b0-9c78-b4b960c5369f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:46.474779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1071/2123] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:46 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:46.479390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df924459-c323-4119-8760-85e88b0e197b admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:46.490032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df924459-c323-4119-8760-85e88b0e197b admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:46.495001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a712ab6-42ea-457c-8aac-6d7391222a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:46.495593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a712ab6-42ea-457c-8aac-6d7391222a2a admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.495872 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a712ab6-42ea-457c-8aac-6d7391222a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.496140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a712ab6-42ea-457c-8aac-6d7391222a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.496828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a712ab6-42ea-457c-8aac-6d7391222a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:46.498475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a712ab6-42ea-457c-8aac-6d7391222a2a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:46.508236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1053/2124] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:47:46.512012 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9714e-a0bb-44b0-9c78-b4b960c5369f admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:46.522482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9714e-a0bb-44b0-9c78-b4b960c5369f admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:46.527343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f281760-7aba-4b8d-9c69-a4816b97eb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:46.527967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f281760-7aba-4b8d-9c69-a4816b97eb84 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.528363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f281760-7aba-4b8d-9c69-a4816b97eb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.528761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f281760-7aba-4b8d-9c69-a4816b97eb84 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.529432 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f281760-7aba-4b8d-9c69-a4816b97eb84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:46.531106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f281760-7aba-4b8d-9c69-a4816b97eb84 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:46.542189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1072/2125] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:46.546181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a712ab6-42ea-457c-8aac-6d7391222a2a admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:46.555837 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a712ab6-42ea-457c-8aac-6d7391222a2a admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:46.560401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde84aa-0599-4008-9674-61de1c1309b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:46.561024 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde84aa-0599-4008-9674-61de1c1309b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.561301 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde84aa-0599-4008-9674-61de1c1309b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.561864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde84aa-0599-4008-9674-61de1c1309b2 admin admin] PATH_INFO: `/v3/projects/cfb53bab3dfc459797187b99b20edd43/users/ef0630a649a0421e9f029cc62f1aa7d6/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.575560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde84aa-0599-4008-9674-61de1c1309b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb53bab3dfc459797187b99b20edd43, user_id=ef0630a649a0421e9f029cc62f1aa7d6, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:46.578230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde84aa-0599-4008-9674-61de1c1309b2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:46.591392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1054/2126] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:46 2022] PUT /identity/v3/projects/cfb53bab3dfc459797187b99b20edd43/users/ef0630a649a0421e9f029cc62f1aa7d6/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:46.602258 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33fe14-ea20-438b-a7d2-ee952fd44250 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.602548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33fe14-ea20-438b-a7d2-ee952fd44250 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.602824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33fe14-ea20-438b-a7d2-ee952fd44250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.630397 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-3c33fe14-ea20-438b-a7d2-ee952fd44250 None None] MFA Rules not processed for user `ef0630a649a0421e9f029cc62f1aa7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:46.631606 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3c33fe14-ea20-438b-a7d2-ee952fd44250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:46.696725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1073/2127] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:46.701221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde84aa-0599-4008-9674-61de1c1309b2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:46.722835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:46.766278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde84aa-0599-4008-9674-61de1c1309b2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:46.770940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2de3b6-4691-4a7a-924f-49f09f2bf7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:46.771544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2de3b6-4691-4a7a-924f-49f09f2bf7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.772189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2de3b6-4691-4a7a-924f-49f09f2bf7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.772189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2de3b6-4691-4a7a-924f-49f09f2bf7a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.772709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2de3b6-4691-4a7a-924f-49f09f2bf7a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:46.774627 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2de3b6-4691-4a7a-924f-49f09f2bf7a1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:46.802132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1055/2128] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:46 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:46.806156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33fe14-ea20-438b-a7d2-ee952fd44250 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:46.817052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33fe14-ea20-438b-a7d2-ee952fd44250 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:46.829190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:46.873308 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5eddcc-4731-4ec0-b8a1-467d52695251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:46.873936 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5eddcc-4731-4ec0-b8a1-467d52695251 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.874219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5eddcc-4731-4ec0-b8a1-467d52695251 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.874491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5eddcc-4731-4ec0-b8a1-467d52695251 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.875117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5eddcc-4731-4ec0-b8a1-467d52695251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:46.878857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5eddcc-4731-4ec0-b8a1-467d52695251 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:46.903031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1074/2129] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:46 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:46.906940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2de3b6-4691-4a7a-924f-49f09f2bf7a1 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:46.917845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2de3b6-4691-4a7a-924f-49f09f2bf7a1 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:46.923075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0fc57-a53d-402d-a401-4f77457bc067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:46.923672 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0fc57-a53d-402d-a401-4f77457bc067 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.923950 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0fc57-a53d-402d-a401-4f77457bc067 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.924216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0fc57-a53d-402d-a401-4f77457bc067 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.924847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0fc57-a53d-402d-a401-4f77457bc067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:46.926544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0fc57-a53d-402d-a401-4f77457bc067 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:46.941497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1056/2130] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:46.942924 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5eddcc-4731-4ec0-b8a1-467d52695251 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:46.954565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5eddcc-4731-4ec0-b8a1-467d52695251 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:46.960088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4e4f2-d6ff-47d2-a8c2-97394bbd29b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:46.960890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4e4f2-d6ff-47d2-a8c2-97394bbd29b3 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.961172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4e4f2-d6ff-47d2-a8c2-97394bbd29b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.961442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4e4f2-d6ff-47d2-a8c2-97394bbd29b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:46.962233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4e4f2-d6ff-47d2-a8c2-97394bbd29b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:46.963881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4e4f2-d6ff-47d2-a8c2-97394bbd29b3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:46.974724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1075/2131] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:46.978987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0fc57-a53d-402d-a401-4f77457bc067 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:46.990606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0fc57-a53d-402d-a401-4f77457bc067 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:46.996742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715b15a-ce02-4b2b-a988-492eb233978d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:46.997497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715b15a-ce02-4b2b-a988-492eb233978d admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:46.998183 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715b15a-ce02-4b2b-a988-492eb233978d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:46.998362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715b15a-ce02-4b2b-a988-492eb233978d admin admin] PATH_INFO: `/v3/projects/6717d382b03d4551aaf72a696aa9bfca/users/06be9214c42341b8b093c0d776a50302/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:47.014865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715b15a-ce02-4b2b-a988-492eb233978d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6717d382b03d4551aaf72a696aa9bfca, user_id=06be9214c42341b8b093c0d776a50302, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:47.017439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715b15a-ce02-4b2b-a988-492eb233978d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:47.041568 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1057/2132] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:46 2022] PUT /identity/v3/projects/6717d382b03d4551aaf72a696aa9bfca/users/06be9214c42341b8b093c0d776a50302/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:47.044594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77286c06-1ea2-4ffc-a986-79b2f1e4ed2d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:47.044888 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77286c06-1ea2-4ffc-a986-79b2f1e4ed2d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:47.045166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77286c06-1ea2-4ffc-a986-79b2f1e4ed2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:47.078165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-77286c06-1ea2-4ffc-a986-79b2f1e4ed2d None None] MFA Rules not processed for user `06be9214c42341b8b093c0d776a50302`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:47.079677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-77286c06-1ea2-4ffc-a986-79b2f1e4ed2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:47.165980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1076/2133] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:47.169726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715b15a-ce02-4b2b-a988-492eb233978d admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:47.239399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715b15a-ce02-4b2b-a988-492eb233978d admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:47.249975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c8a35-7063-4963-92f4-b79ee3859c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:47.251044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c8a35-7063-4963-92f4-b79ee3859c2e admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:47.251717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c8a35-7063-4963-92f4-b79ee3859c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:47.252244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c8a35-7063-4963-92f4-b79ee3859c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:47.253353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c8a35-7063-4963-92f4-b79ee3859c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:47.255767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c8a35-7063-4963-92f4-b79ee3859c2e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:47.297605 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1058/2134] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:47 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:47.302288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77286c06-1ea2-4ffc-a986-79b2f1e4ed2d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:47.329312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77286c06-1ea2-4ffc-a986-79b2f1e4ed2d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:47.398210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be534c-094d-47af-bfc0-58cd829845d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:47.398821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be534c-094d-47af-bfc0-58cd829845d2 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:47.399101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be534c-094d-47af-bfc0-58cd829845d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:47.399372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be534c-094d-47af-bfc0-58cd829845d2 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:47.399988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be534c-094d-47af-bfc0-58cd829845d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:47.401941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be534c-094d-47af-bfc0-58cd829845d2 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:47.423434 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1077/2135] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:47 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:47.427545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c8a35-7063-4963-92f4-b79ee3859c2e admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:47.438087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c8a35-7063-4963-92f4-b79ee3859c2e admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:47.442696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4e41a-66cc-48f5-9fc7-1b5018476d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:47.443318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4e41a-66cc-48f5-9fc7-1b5018476d5d admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:47.443596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4e41a-66cc-48f5-9fc7-1b5018476d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:47.443865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4e41a-66cc-48f5-9fc7-1b5018476d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:47.444502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4e41a-66cc-48f5-9fc7-1b5018476d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:47.446120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4e41a-66cc-48f5-9fc7-1b5018476d5d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:47.457847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1059/2136] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:47.461793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be534c-094d-47af-bfc0-58cd829845d2 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:47.471939 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be534c-094d-47af-bfc0-58cd829845d2 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:47.477147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1592d-2af1-4f9a-9727-a621c0ca32a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:47.477820 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1592d-2af1-4f9a-9727-a621c0ca32a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:47.478097 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1592d-2af1-4f9a-9727-a621c0ca32a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:47.478366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1592d-2af1-4f9a-9727-a621c0ca32a8 admin admin] PATH_INFO: `/v3/projects/0f50baa8dcf348c4ac8def8dddd769ed/users/95ef5542f96444279ea59ff27ec7a067/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:47.490002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:47.493196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1592d-2af1-4f9a-9727-a621c0ca32a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f50baa8dcf348c4ac8def8dddd769ed, user_id=95ef5542f96444279ea59ff27ec7a067, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:47.496248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1592d-2af1-4f9a-9727-a621c0ca32a8 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:47.509010 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1078/2137] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:47 2022] PUT /identity/v3/projects/0f50baa8dcf348c4ac8def8dddd769ed/users/95ef5542f96444279ea59ff27ec7a067/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:47.512965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4e41a-66cc-48f5-9fc7-1b5018476d5d admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:47.523447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4e41a-66cc-48f5-9fc7-1b5018476d5d admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:47.565995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:47.581294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ee33a-fabe-4ba7-9293-c098d4de3777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:47.581939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ee33a-fabe-4ba7-9293-c098d4de3777 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:47.582219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ee33a-fabe-4ba7-9293-c098d4de3777 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:47.582490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ee33a-fabe-4ba7-9293-c098d4de3777 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:47.583126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ee33a-fabe-4ba7-9293-c098d4de3777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:47.584720 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ee33a-fabe-4ba7-9293-c098d4de3777 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:47.594621 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1060/2138] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:47.599298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1592d-2af1-4f9a-9727-a621c0ca32a8 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:47.609271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1592d-2af1-4f9a-9727-a621c0ca32a8 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:47.614342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ada4dc-bc66-490e-a7fd-80032fe87b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:47.614984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ada4dc-bc66-490e-a7fd-80032fe87b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:47.615259 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ada4dc-bc66-490e-a7fd-80032fe87b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:47.615528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ada4dc-bc66-490e-a7fd-80032fe87b82 admin admin] PATH_INFO: `/v3/projects/0f50baa8dcf348c4ac8def8dddd769ed/users/95ef5542f96444279ea59ff27ec7a067/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:47.617391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ada4dc-bc66-490e-a7fd-80032fe87b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f50baa8dcf348c4ac8def8dddd769ed, user_id=95ef5542f96444279ea59ff27ec7a067, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:47.620327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ada4dc-bc66-490e-a7fd-80032fe87b82 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:47.630184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1079/2139] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:47 2022] PUT /identity/v3/projects/0f50baa8dcf348c4ac8def8dddd769ed/users/95ef5542f96444279ea59ff27ec7a067/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:47.640649 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc16624-668a-41d7-82b0-2b144e778527 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:47.640922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc16624-668a-41d7-82b0-2b144e778527 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:47.641201 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc16624-668a-41d7-82b0-2b144e778527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:47.670260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-fbc16624-668a-41d7-82b0-2b144e778527 None None] MFA Rules not processed for user `95ef5542f96444279ea59ff27ec7a067`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:47.671500 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-fbc16624-668a-41d7-82b0-2b144e778527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:47.744343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1061/2140] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:47:47.755910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ada4dc-bc66-490e-a7fd-80032fe87b82 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:47.757648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-14ada4dc-bc66-490e-a7fd-80032fe87b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:47.814316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ada4dc-bc66-490e-a7fd-80032fe87b82 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:47.818025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d38dfc-e77b-442c-98f4-44fc45568241 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:47.818657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d38dfc-e77b-442c-98f4-44fc45568241 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:47.818937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d38dfc-e77b-442c-98f4-44fc45568241 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:47.819205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d38dfc-e77b-442c-98f4-44fc45568241 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:47.873559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d38dfc-e77b-442c-98f4-44fc45568241 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:47.875016 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d38dfc-e77b-442c-98f4-44fc45568241 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:47.883629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1080/2141] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:49.409196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc16624-668a-41d7-82b0-2b144e778527 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:49.411281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-fbc16624-668a-41d7-82b0-2b144e778527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:49.423469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc16624-668a-41d7-82b0-2b144e778527 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:49.427152 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a28aad-72f6-4ecc-aaa6-8f122b98f558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:49.427799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a28aad-72f6-4ecc-aaa6-8f122b98f558 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:49.428174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a28aad-72f6-4ecc-aaa6-8f122b98f558 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:49.428332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a28aad-72f6-4ecc-aaa6-8f122b98f558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:49.490230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a28aad-72f6-4ecc-aaa6-8f122b98f558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:49.491667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a28aad-72f6-4ecc-aaa6-8f122b98f558 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:49.500756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1062/2142] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:50.127741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d38dfc-e77b-442c-98f4-44fc45568241 service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:50.136374 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d38dfc-e77b-442c-98f4-44fc45568241 service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:50.139964 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef15861-22ca-4281-88b1-5f0b7cc2e933 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:50.140563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef15861-22ca-4281-88b1-5f0b7cc2e933 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:50.140844 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef15861-22ca-4281-88b1-5f0b7cc2e933 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:50.141176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef15861-22ca-4281-88b1-5f0b7cc2e933 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:50.150470 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef15861-22ca-4281-88b1-5f0b7cc2e933 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:50.151910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef15861-22ca-4281-88b1-5f0b7cc2e933 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:50.162162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1081/2143] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:50.364773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a28aad-72f6-4ecc-aaa6-8f122b98f558 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:50.366213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-17a28aad-72f6-4ecc-aaa6-8f122b98f558 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:50.431550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a28aad-72f6-4ecc-aaa6-8f122b98f558 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:50.436332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:50.436951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:50.437227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:50.437494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] PATH_INFO: `/v3/users/9c261ef4393844d88d571a558eddcb40` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:50.438628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c261ef4393844d88d571a558eddcb40)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:50.440566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:50.445387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:50.490654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c261ef4393844d88d571a558eddcb40'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:50.491252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c261ef4393844d88d571a558eddcb40'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:50.512207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c261ef4393844d88d571a558eddcb40'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:50.523396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c261ef4393844d88d571a558eddcb40'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:50.524835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c261ef4393844d88d571a558eddcb40'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:50.539346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1063/2144] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:50 2022] DELETE /identity/v3/users/9c261ef4393844d88d571a558eddcb40 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:50.543724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadd963-d579-47e3-8e44-39a4a39f435f None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:50.544013 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadd963-d579-47e3-8e44-39a4a39f435f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:50.544317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadd963-d579-47e3-8e44-39a4a39f435f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:50.564951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-fdadd963-d579-47e3-8e44-39a4a39f435f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:50.566743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fdadd963-d579-47e3-8e44-39a4a39f435f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:50.626251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1082/2145] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:50.636460 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:50.637881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:50.705325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c286cf-e6ee-495b-b847-016d72e92638 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:50.709047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9825c4-d2cf-471c-9e1d-632cd5065804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:50.709660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9825c4-d2cf-471c-9e1d-632cd5065804 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:50.709968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9825c4-d2cf-471c-9e1d-632cd5065804 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:50.710244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9825c4-d2cf-471c-9e1d-632cd5065804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:50.719590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9825c4-d2cf-471c-9e1d-632cd5065804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:50.721096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9825c4-d2cf-471c-9e1d-632cd5065804 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:50.731703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1064/2146] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:47:51.064132 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadd963-d579-47e3-8e44-39a4a39f435f None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:51.067205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fdadd963-d579-47e3-8e44-39a4a39f435f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:51.082454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadd963-d579-47e3-8e44-39a4a39f435f None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:51.087676 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:51.088321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:51.088632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:51.088880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79 admin admin] PATH_INFO: `/v3/projects/3f68842ccdfa4382a3dda6b607fa90e2` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:51.090252 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f68842ccdfa4382a3dda6b607fa90e2)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:51.092540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:51.112303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:51.153448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f68842ccdfa4382a3dda6b607fa90e2', 'request_id': 'req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:51.160832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f68842ccdfa4382a3dda6b607fa90e2', 'request_id': 'req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:51.166809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f68842ccdfa4382a3dda6b607fa90e2', 'request_id': 'req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:51.180658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79 admin admin] The token cache is being invalidate because project 3f68842ccdfa4382a3dda6b607fa90e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:47:51.181033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:51.181659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-586afe14-ddcb-4f02-bbc3-e1b9a5c99f79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:51.183336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1083/2147] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:47:51 2022] DELETE /identity/v3/projects/3f68842ccdfa4382a3dda6b607fa90e2 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:51.188308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca6cee-1894-4b99-ae2f-59a4f1d64174 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:51.188598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca6cee-1894-4b99-ae2f-59a4f1d64174 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:51.188978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca6cee-1894-4b99-ae2f-59a4f1d64174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:51.212818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-3eca6cee-1894-4b99-ae2f-59a4f1d64174 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:51.214101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3eca6cee-1894-4b99-ae2f-59a4f1d64174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:51.278392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1065/2148] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:51.286357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75fa2b-b550-4978-bc7d-59d1a00989cc None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:51.286646 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75fa2b-b550-4978-bc7d-59d1a00989cc None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:51.286920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75fa2b-b550-4978-bc7d-59d1a00989cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:51.311524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-ab75fa2b-b550-4978-bc7d-59d1a00989cc None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:51.312726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ab75fa2b-b550-4978-bc7d-59d1a00989cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:51.322050 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1084/2149] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:51.326596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca6cee-1894-4b99-ae2f-59a4f1d64174 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:51.338211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca6cee-1894-4b99-ae2f-59a4f1d64174 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:51.343116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a51fdd-c6ca-4930-a3db-4c6c1b352734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:51.343995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a51fdd-c6ca-4930-a3db-4c6c1b352734 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:51.344283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a51fdd-c6ca-4930-a3db-4c6c1b352734 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:51.344556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a51fdd-c6ca-4930-a3db-4c6c1b352734 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:51.345198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a51fdd-c6ca-4930-a3db-4c6c1b352734 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:47:51.345506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a51fdd-c6ca-4930-a3db-4c6c1b352734 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:51.347184 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a51fdd-c6ca-4930-a3db-4c6c1b352734 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:51.359022 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1066/2150] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:47:51.363518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75fa2b-b550-4978-bc7d-59d1a00989cc None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:51.373273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75fa2b-b550-4978-bc7d-59d1a00989cc None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:51.378738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e648b-a709-492f-b6bf-b9a5949051ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:51.379372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e648b-a709-492f-b6bf-b9a5949051ec admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:51.379657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e648b-a709-492f-b6bf-b9a5949051ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:51.379931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e648b-a709-492f-b6bf-b9a5949051ec admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:51.380576 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e648b-a709-492f-b6bf-b9a5949051ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:51.382523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e648b-a709-492f-b6bf-b9a5949051ec admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:51.410898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1085/2151] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:51 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:51.415146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a51fdd-c6ca-4930-a3db-4c6c1b352734 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:51.426494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a51fdd-c6ca-4930-a3db-4c6c1b352734 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:51.437551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:51.491649 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d75d0-bc66-48ed-9bc9-8d9adb5a12ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:51.492494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d75d0-bc66-48ed-9bc9-8d9adb5a12ac admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:51.492800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d75d0-bc66-48ed-9bc9-8d9adb5a12ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:51.493105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d75d0-bc66-48ed-9bc9-8d9adb5a12ac admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:51.493796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d75d0-bc66-48ed-9bc9-8d9adb5a12ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:51.495940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d75d0-bc66-48ed-9bc9-8d9adb5a12ac admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:51.532244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1067/2152] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:51 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:51.532694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e648b-a709-492f-b6bf-b9a5949051ec admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:51.536772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e648b-a709-492f-b6bf-b9a5949051ec admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:51.541910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d41d9b-9c85-46b9-a082-561f85a8519e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:51.542607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d41d9b-9c85-46b9-a082-561f85a8519e admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:51.542984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d41d9b-9c85-46b9-a082-561f85a8519e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:51.543323 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d41d9b-9c85-46b9-a082-561f85a8519e admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:51.544074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d41d9b-9c85-46b9-a082-561f85a8519e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:51.545672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d41d9b-9c85-46b9-a082-561f85a8519e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:51.556426 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1086/2153] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:47:51.560177 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d75d0-bc66-48ed-9bc9-8d9adb5a12ac admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:51.575607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d75d0-bc66-48ed-9bc9-8d9adb5a12ac admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:51.582581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0854c-ddf1-4f2e-9824-38e70564e848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:51.583317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0854c-ddf1-4f2e-9824-38e70564e848 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:51.583695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0854c-ddf1-4f2e-9824-38e70564e848 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:51.584066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0854c-ddf1-4f2e-9824-38e70564e848 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:51.584805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0854c-ddf1-4f2e-9824-38e70564e848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:51.586588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0854c-ddf1-4f2e-9824-38e70564e848 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:51.598182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1068/2154] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:51.602224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d41d9b-9c85-46b9-a082-561f85a8519e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:51.612880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d41d9b-9c85-46b9-a082-561f85a8519e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:51.617822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b08e09-c11a-412e-813d-72edfb2ef505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:51.618486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b08e09-c11a-412e-813d-72edfb2ef505 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:51.618764 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b08e09-c11a-412e-813d-72edfb2ef505 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:51.619037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b08e09-c11a-412e-813d-72edfb2ef505 admin admin] PATH_INFO: `/v3/projects/fe42bae7e14246d78103a4ef69457051/users/8fcf4142aa8340d692b1b3ed0aee7c87/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:51.634831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b08e09-c11a-412e-813d-72edfb2ef505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe42bae7e14246d78103a4ef69457051, user_id=8fcf4142aa8340d692b1b3ed0aee7c87, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:51.637681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b08e09-c11a-412e-813d-72edfb2ef505 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:51.650984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1087/2155] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:51 2022] PUT /identity/v3/projects/fe42bae7e14246d78103a4ef69457051/users/8fcf4142aa8340d692b1b3ed0aee7c87/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:47:51.656099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb44bf6-5f3f-4965-9865-66b0b16277ac None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:51.656381 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb44bf6-5f3f-4965-9865-66b0b16277ac None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:51.656674 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb44bf6-5f3f-4965-9865-66b0b16277ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:51.677503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-8cb44bf6-5f3f-4965-9865-66b0b16277ac None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:51.678724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8cb44bf6-5f3f-4965-9865-66b0b16277ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:51.735681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1069/2156] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:51.745234 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b08e09-c11a-412e-813d-72edfb2ef505 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:51.746976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b1b08e09-c11a-412e-813d-72edfb2ef505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:51.803591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b08e09-c11a-412e-813d-72edfb2ef505 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:51.807207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f55c53-ee96-44f9-b97d-6440a435c8c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:51.807802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f55c53-ee96-44f9-b97d-6440a435c8c8 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:51.808082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f55c53-ee96-44f9-b97d-6440a435c8c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:51.808354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f55c53-ee96-44f9-b97d-6440a435c8c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:51.817862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f55c53-ee96-44f9-b97d-6440a435c8c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:51.819294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f55c53-ee96-44f9-b97d-6440a435c8c8 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:51.828692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1088/2157] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:52.035035 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb44bf6-5f3f-4965-9865-66b0b16277ac None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:52.036679 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8cb44bf6-5f3f-4965-9865-66b0b16277ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:52.048772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb44bf6-5f3f-4965-9865-66b0b16277ac None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:52.054420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:52.055035 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:52.055314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:52.055585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] PATH_INFO: `/v3/users/0c4b6c7c115e4050aa879477878a0e5c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:52.056744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c4b6c7c115e4050aa879477878a0e5c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:52.058684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:52.106677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c4b6c7c115e4050aa879477878a0e5c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:52.107282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c4b6c7c115e4050aa879477878a0e5c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:52.120185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c4b6c7c115e4050aa879477878a0e5c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:52.127117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c4b6c7c115e4050aa879477878a0e5c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:52.127472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c4b6c7c115e4050aa879477878a0e5c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:52.141054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1070/2158] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:52 2022] DELETE /identity/v3/users/0c4b6c7c115e4050aa879477878a0e5c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:52.147272 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863256d-e020-4b45-ae4b-316da47bfe85 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:52.147668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863256d-e020-4b45-ae4b-316da47bfe85 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:52.147978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863256d-e020-4b45-ae4b-316da47bfe85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:52.179101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6863256d-e020-4b45-ae4b-316da47bfe85 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:52.180377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6863256d-e020-4b45-ae4b-316da47bfe85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:52.240406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:52.270052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1089/2159] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:52.281246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:52.283085 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:52.292638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:52.378012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a2e04-bfbe-4983-8173-97ea290ba468 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:52.382261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd1924-b961-4ad4-ba52-dfee4c69ff05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:52.382863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd1924-b961-4ad4-ba52-dfee4c69ff05 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:52.383141 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd1924-b961-4ad4-ba52-dfee4c69ff05 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:52.383484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd1924-b961-4ad4-ba52-dfee4c69ff05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:52.394098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd1924-b961-4ad4-ba52-dfee4c69ff05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:52.395342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd1924-b961-4ad4-ba52-dfee4c69ff05 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:52.405863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1071/2160] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:52.949111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863256d-e020-4b45-ae4b-316da47bfe85 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:52.951268 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6863256d-e020-4b45-ae4b-316da47bfe85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:52.966783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863256d-e020-4b45-ae4b-316da47bfe85 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:52.973506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc618a85-11a5-410f-835f-ffa9ae7718d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:52.974699 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc618a85-11a5-410f-835f-ffa9ae7718d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:52.975507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc618a85-11a5-410f-835f-ffa9ae7718d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:52.975932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc618a85-11a5-410f-835f-ffa9ae7718d0 admin admin] PATH_INFO: `/v3/projects/a23ad8e74b2e4740aaa6437312d0ee61` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:52.977509 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc618a85-11a5-410f-835f-ffa9ae7718d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a23ad8e74b2e4740aaa6437312d0ee61)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:52.982095 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc618a85-11a5-410f-835f-ffa9ae7718d0 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:53.058358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-cc618a85-11a5-410f-835f-ffa9ae7718d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a23ad8e74b2e4740aaa6437312d0ee61', 'request_id': 'req-cc618a85-11a5-410f-835f-ffa9ae7718d0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:53.072629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-cc618a85-11a5-410f-835f-ffa9ae7718d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a23ad8e74b2e4740aaa6437312d0ee61', 'request_id': 'req-cc618a85-11a5-410f-835f-ffa9ae7718d0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:53.078846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-cc618a85-11a5-410f-835f-ffa9ae7718d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a23ad8e74b2e4740aaa6437312d0ee61', 'request_id': 'req-cc618a85-11a5-410f-835f-ffa9ae7718d0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:53.094974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-cc618a85-11a5-410f-835f-ffa9ae7718d0 admin admin] The token cache is being invalidate because project a23ad8e74b2e4740aaa6437312d0ee61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:47:53.095281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-cc618a85-11a5-410f-835f-ffa9ae7718d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:53.095912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-cc618a85-11a5-410f-835f-ffa9ae7718d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:53.098086 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1090/2161] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:47:52 2022] DELETE /identity/v3/projects/a23ad8e74b2e4740aaa6437312d0ee61 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:53.103171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5ba37-2da1-4bf7-88bd-15f3e091ac73 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:53.103652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5ba37-2da1-4bf7-88bd-15f3e091ac73 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:53.103731 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5ba37-2da1-4bf7-88bd-15f3e091ac73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:53.133075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a1d5ba37-2da1-4bf7-88bd-15f3e091ac73 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:53.134305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a1d5ba37-2da1-4bf7-88bd-15f3e091ac73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:53.228597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1072/2162] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:53.238629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d7362-3abb-4ec3-ba27-5e8239cadfac None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:53.239210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d7362-3abb-4ec3-ba27-5e8239cadfac None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:53.239210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d7362-3abb-4ec3-ba27-5e8239cadfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:53.265595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-3d6d7362-3abb-4ec3-ba27-5e8239cadfac None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:53.277290 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3d6d7362-3abb-4ec3-ba27-5e8239cadfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:53.303137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1091/2163] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:53.307604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5ba37-2da1-4bf7-88bd-15f3e091ac73 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:53.321351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5ba37-2da1-4bf7-88bd-15f3e091ac73 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:53.328290 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9b662-02e9-46f6-9e82-11933eb399fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:53.329419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9b662-02e9-46f6-9e82-11933eb399fd admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:53.329943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9b662-02e9-46f6-9e82-11933eb399fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:53.330808 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9b662-02e9-46f6-9e82-11933eb399fd admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:53.332203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9b662-02e9-46f6-9e82-11933eb399fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:47:53.332843 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9b662-02e9-46f6-9e82-11933eb399fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:53.335164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9b662-02e9-46f6-9e82-11933eb399fd admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:53.360564 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1073/2164] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:47:53.366215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d7362-3abb-4ec3-ba27-5e8239cadfac None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:53.386798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d7362-3abb-4ec3-ba27-5e8239cadfac None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:53.392576 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd06cb-3e29-41bc-9d9e-020e947ecbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:53.393556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd06cb-3e29-41bc-9d9e-020e947ecbfa admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:53.394022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd06cb-3e29-41bc-9d9e-020e947ecbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:53.394446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd06cb-3e29-41bc-9d9e-020e947ecbfa admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:53.398928 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd06cb-3e29-41bc-9d9e-020e947ecbfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:53.401096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd06cb-3e29-41bc-9d9e-020e947ecbfa admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:53.438352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1092/2165] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:53 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:47:53.443410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9b662-02e9-46f6-9e82-11933eb399fd admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:53.459689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9b662-02e9-46f6-9e82-11933eb399fd admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:53.525038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce48fd-c66e-4f55-afcb-e5236b932c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:53.525651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce48fd-c66e-4f55-afcb-e5236b932c59 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:53.525976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce48fd-c66e-4f55-afcb-e5236b932c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:53.526248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce48fd-c66e-4f55-afcb-e5236b932c59 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:53.527046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce48fd-c66e-4f55-afcb-e5236b932c59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:53.528859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce48fd-c66e-4f55-afcb-e5236b932c59 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:53.570674 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1074/2166] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:53 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:53.575038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd06cb-3e29-41bc-9d9e-020e947ecbfa admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:53.586456 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd06cb-3e29-41bc-9d9e-020e947ecbfa admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:53.600262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc695fda-2794-4b3f-b45f-15a419d96f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:53.601138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc695fda-2794-4b3f-b45f-15a419d96f62 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:53.601520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc695fda-2794-4b3f-b45f-15a419d96f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:53.601924 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc695fda-2794-4b3f-b45f-15a419d96f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:53.602873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc695fda-2794-4b3f-b45f-15a419d96f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:53.604804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc695fda-2794-4b3f-b45f-15a419d96f62 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:53.616719 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1093/2167] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:47:53.625166 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce48fd-c66e-4f55-afcb-e5236b932c59 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:53.658093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce48fd-c66e-4f55-afcb-e5236b932c59 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:53.674634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257b2d8-ca41-4e67-bc94-c84b6bab3bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:53.676057 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b2d8-ca41-4e67-bc94-c84b6bab3bdf admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:53.676198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b2d8-ca41-4e67-bc94-c84b6bab3bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:53.676479 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257b2d8-ca41-4e67-bc94-c84b6bab3bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:53.677129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257b2d8-ca41-4e67-bc94-c84b6bab3bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:53.678817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257b2d8-ca41-4e67-bc94-c84b6bab3bdf admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:53.699701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1075/2168] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:53.705274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc695fda-2794-4b3f-b45f-15a419d96f62 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:53.723866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc695fda-2794-4b3f-b45f-15a419d96f62 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:53.729187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a4aae-d3fb-441f-808c-cb7d780c99b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:53.730087 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a4aae-d3fb-441f-808c-cb7d780c99b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:53.730517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a4aae-d3fb-441f-808c-cb7d780c99b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:53.730966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a4aae-d3fb-441f-808c-cb7d780c99b4 admin admin] PATH_INFO: `/v3/projects/34f73eeee4f145e7981f4b88a0fa23b8/users/f799b8e0fb4f44eea4230e4925f0ed2b/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:53.757078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a4aae-d3fb-441f-808c-cb7d780c99b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f73eeee4f145e7981f4b88a0fa23b8, user_id=f799b8e0fb4f44eea4230e4925f0ed2b, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:53.760187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a4aae-d3fb-441f-808c-cb7d780c99b4 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:53.780854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1094/2169] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:53 2022] PUT /identity/v3/projects/34f73eeee4f145e7981f4b88a0fa23b8/users/f799b8e0fb4f44eea4230e4925f0ed2b/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:53.794910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8746d-89c5-4081-840b-abc0ad17b8da None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:53.795510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8746d-89c5-4081-840b-abc0ad17b8da None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:53.796141 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8746d-89c5-4081-840b-abc0ad17b8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:53.850089 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:53.857045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-39b8746d-89c5-4081-840b-abc0ad17b8da None None] MFA Rules not processed for user `f799b8e0fb4f44eea4230e4925f0ed2b`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:53.861869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-39b8746d-89c5-4081-840b-abc0ad17b8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:53.989451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1076/2170] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:54.013481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a4aae-d3fb-441f-808c-cb7d780c99b4 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:54.015457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2d8a4aae-d3fb-441f-808c-cb7d780c99b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:54.123300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a4aae-d3fb-441f-808c-cb7d780c99b4 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:54.126112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6314a02-0761-4a95-ba9d-a6a745ee7b34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:54.126839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6314a02-0761-4a95-ba9d-a6a745ee7b34 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:54.127739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6314a02-0761-4a95-ba9d-a6a745ee7b34 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:54.128057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6314a02-0761-4a95-ba9d-a6a745ee7b34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:54.141739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6314a02-0761-4a95-ba9d-a6a745ee7b34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:54.144163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6314a02-0761-4a95-ba9d-a6a745ee7b34 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:54.155082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1095/2171] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:54.165410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8746d-89c5-4081-840b-abc0ad17b8da None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:54.166919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-39b8746d-89c5-4081-840b-abc0ad17b8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:54.259612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8746d-89c5-4081-840b-abc0ad17b8da None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:54.264492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:54.265398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:54.265865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:54.266293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] PATH_INFO: `/v3/users/ef0630a649a0421e9f029cc62f1aa7d6` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:54.268982 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef0630a649a0421e9f029cc62f1aa7d6)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:54.271226 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:54.347958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef0630a649a0421e9f029cc62f1aa7d6'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:54.349761 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef0630a649a0421e9f029cc62f1aa7d6'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:54.365562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef0630a649a0421e9f029cc62f1aa7d6'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:54.374468 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef0630a649a0421e9f029cc62f1aa7d6'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:54.375345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef0630a649a0421e9f029cc62f1aa7d6'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:54.397911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1077/2172] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:54 2022] DELETE /identity/v3/users/ef0630a649a0421e9f029cc62f1aa7d6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:54.402517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89e1c9-ff40-4793-8ea9-66e39606784a None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:54.403347 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89e1c9-ff40-4793-8ea9-66e39606784a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:54.403772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89e1c9-ff40-4793-8ea9-66e39606784a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:54.414556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:54.434240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-8e89e1c9-ff40-4793-8ea9-66e39606784a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:54.435578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8e89e1c9-ff40-4793-8ea9-66e39606784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:54.546406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1096/2173] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:54.569881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:54.569881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:54.608555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:54.670298 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fbac2-5297-4a07-8f73-933009e52f72 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:54.673987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8d78e-7da2-45b1-aa44-c46c9ad13c87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:54.674865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8d78e-7da2-45b1-aa44-c46c9ad13c87 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:54.675029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8d78e-7da2-45b1-aa44-c46c9ad13c87 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:54.675859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff8d78e-7da2-45b1-aa44-c46c9ad13c87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:54.687333 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8d78e-7da2-45b1-aa44-c46c9ad13c87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:54.689342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff8d78e-7da2-45b1-aa44-c46c9ad13c87 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:54.702308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1078/2174] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:55.071919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89e1c9-ff40-4793-8ea9-66e39606784a None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:55.074028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8e89e1c9-ff40-4793-8ea9-66e39606784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:55.088524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89e1c9-ff40-4793-8ea9-66e39606784a None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:55.105365 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:55.106059 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:55.106339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:55.106612 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] PATH_INFO: `/v3/projects/cfb53bab3dfc459797187b99b20edd43` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:55.108957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfb53bab3dfc459797187b99b20edd43)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:55.111176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:55.197479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb53bab3dfc459797187b99b20edd43', 'request_id': 'req-1de92be9-77dc-4c42-9368-941f44e63190'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:55.204675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb53bab3dfc459797187b99b20edd43', 'request_id': 'req-1de92be9-77dc-4c42-9368-941f44e63190'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:55.212987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb53bab3dfc459797187b99b20edd43', 'request_id': 'req-1de92be9-77dc-4c42-9368-941f44e63190'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:55.230309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] The token cache is being invalidate because project cfb53bab3dfc459797187b99b20edd43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:47:55.231454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:55.232375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:55.234264 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1097/2175] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:47:55 2022] DELETE /identity/v3/projects/cfb53bab3dfc459797187b99b20edd43 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:55.239665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8d78e-7da2-45b1-aa44-c46c9ad13c87 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:55.251591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff8d78e-7da2-45b1-aa44-c46c9ad13c87 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:55.346012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:55.347282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:55.347566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:55.347840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] PATH_INFO: `/v3/users/06be9214c42341b8b093c0d776a50302` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:55.349144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06be9214c42341b8b093c0d776a50302)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:55.351267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:55.410370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06be9214c42341b8b093c0d776a50302'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:55.411463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06be9214c42341b8b093c0d776a50302'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:55.436063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06be9214c42341b8b093c0d776a50302'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:55.445889 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06be9214c42341b8b093c0d776a50302'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:55.446939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06be9214c42341b8b093c0d776a50302'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:55.463736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1079/2176] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:55 2022] DELETE /identity/v3/users/06be9214c42341b8b093c0d776a50302 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:47:56.275086 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:56.279399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:56.317551 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de92be9-77dc-4c42-9368-941f44e63190 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:56.407812 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:56.409774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:56.410070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:56.410354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] PATH_INFO: `/v3/projects/6717d382b03d4551aaf72a696aa9bfca` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:56.412972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6717d382b03d4551aaf72a696aa9bfca)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:56.416427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:56.471451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:56.493223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6717d382b03d4551aaf72a696aa9bfca', 'request_id': 'req-737bc707-a3b8-4f8c-8c14-76da52b121f8'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:56.500622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6717d382b03d4551aaf72a696aa9bfca', 'request_id': 'req-737bc707-a3b8-4f8c-8c14-76da52b121f8'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:56.518026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6717d382b03d4551aaf72a696aa9bfca', 'request_id': 'req-737bc707-a3b8-4f8c-8c14-76da52b121f8'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:56.558473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] The token cache is being invalidate because project 6717d382b03d4551aaf72a696aa9bfca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:47:56.558787 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:56.562343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:56.568109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1098/2177] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:47:56 2022] DELETE /identity/v3/projects/6717d382b03d4551aaf72a696aa9bfca => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:56.569147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:56.570690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:56.587882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df98062-123f-42c0-ab09-ab66543f0c52 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:56.648628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:56.688679 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f564b-8a1c-4698-8555-14672f975249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:56.689822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f564b-8a1c-4698-8555-14672f975249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:56.690285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f564b-8a1c-4698-8555-14672f975249 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:56.690700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f564b-8a1c-4698-8555-14672f975249 admin admin] PATH_INFO: `/v3/users/95ef5542f96444279ea59ff27ec7a067` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:56.693946 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f564b-8a1c-4698-8555-14672f975249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ef5542f96444279ea59ff27ec7a067)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:56.696669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f564b-8a1c-4698-8555-14672f975249 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:56.767597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-227f564b-8a1c-4698-8555-14672f975249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ef5542f96444279ea59ff27ec7a067'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:56.768322 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-227f564b-8a1c-4698-8555-14672f975249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ef5542f96444279ea59ff27ec7a067'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:56.787131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-227f564b-8a1c-4698-8555-14672f975249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ef5542f96444279ea59ff27ec7a067'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:56.796654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-227f564b-8a1c-4698-8555-14672f975249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ef5542f96444279ea59ff27ec7a067'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:56.797941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-227f564b-8a1c-4698-8555-14672f975249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ef5542f96444279ea59ff27ec7a067'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:56.813859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1080/2178] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:56 2022] DELETE /identity/v3/users/95ef5542f96444279ea59ff27ec7a067 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:57.188627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:57.194019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:57.236312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bc707-a3b8-4f8c-8c14-76da52b121f8 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:57.324112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:57.324799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:57.324954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:57.325378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586 admin admin] PATH_INFO: `/v3/projects/0f50baa8dcf348c4ac8def8dddd769ed` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:57.326802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f50baa8dcf348c4ac8def8dddd769ed)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:57.329811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:57.407990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f50baa8dcf348c4ac8def8dddd769ed', 'request_id': 'req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:57.416019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f50baa8dcf348c4ac8def8dddd769ed', 'request_id': 'req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:57.426581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f50baa8dcf348c4ac8def8dddd769ed', 'request_id': 'req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:57.457273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586 admin admin] The token cache is being invalidate because project 0f50baa8dcf348c4ac8def8dddd769ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:47:57.457715 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:57.458487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-af4eb233-8cbe-45b6-9e6f-1d1d4d198586 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:57.460326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1099/2179] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:47:57 2022] DELETE /identity/v3/projects/0f50baa8dcf348c4ac8def8dddd769ed => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:57.465475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35112a4-dd3f-4064-818c-5392f3119f16 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:57.466050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35112a4-dd3f-4064-818c-5392f3119f16 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:57.466574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35112a4-dd3f-4064-818c-5392f3119f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:57.495810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d35112a4-dd3f-4064-818c-5392f3119f16 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:57.497467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d35112a4-dd3f-4064-818c-5392f3119f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:57.572036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1081/2180] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:47:57.581375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eca617-b931-4e7b-9e2e-37ff6189420c None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:57.581739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eca617-b931-4e7b-9e2e-37ff6189420c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:57.582027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eca617-b931-4e7b-9e2e-37ff6189420c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:57.626007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-51eca617-b931-4e7b-9e2e-37ff6189420c None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:57.626007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-51eca617-b931-4e7b-9e2e-37ff6189420c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:57.635731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1100/2181] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:57.639846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35112a4-dd3f-4064-818c-5392f3119f16 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:57.651781 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35112a4-dd3f-4064-818c-5392f3119f16 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:57.670493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3ea8b-b4dc-4161-b65b-4b924da1520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:57.671157 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3ea8b-b4dc-4161-b65b-4b924da1520f admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:57.671446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3ea8b-b4dc-4161-b65b-4b924da1520f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:57.671722 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3ea8b-b4dc-4161-b65b-4b924da1520f admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:57.672425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3ea8b-b4dc-4161-b65b-4b924da1520f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:47:57.672833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3ea8b-b4dc-4161-b65b-4b924da1520f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:57.674536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3ea8b-b4dc-4161-b65b-4b924da1520f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:57.703064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1082/2182] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:47:57.708325 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eca617-b931-4e7b-9e2e-37ff6189420c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:57.733536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eca617-b931-4e7b-9e2e-37ff6189420c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:57.748589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3ea8b-b4dc-4161-b65b-4b924da1520f admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:57.749950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f9b60-6cd3-4d37-bd17-82a05df00e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:57.751256 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f9b60-6cd3-4d37-bd17-82a05df00e0a admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:57.751862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f9b60-6cd3-4d37-bd17-82a05df00e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:57.752773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-72b3ea8b-b4dc-4161-b65b-4b924da1520f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:57.755183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f9b60-6cd3-4d37-bd17-82a05df00e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:57.757615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f9b60-6cd3-4d37-bd17-82a05df00e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:57.761752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7f9b60-6cd3-4d37-bd17-82a05df00e0a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:57.782941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3ea8b-b4dc-4161-b65b-4b924da1520f admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:57.790865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ae193-9612-49d9-93e8-16fd04426420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:57.791523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ae193-9612-49d9-93e8-16fd04426420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:57.791803 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ae193-9612-49d9-93e8-16fd04426420 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:57.792075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ae193-9612-49d9-93e8-16fd04426420 admin admin] PATH_INFO: `/v3/users/f799b8e0fb4f44eea4230e4925f0ed2b` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:57.793282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ae193-9612-49d9-93e8-16fd04426420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f799b8e0fb4f44eea4230e4925f0ed2b)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:57.797156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ae193-9612-49d9-93e8-16fd04426420 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:57.831468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1101/2183] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:57 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:57.833079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f9b60-6cd3-4d37-bd17-82a05df00e0a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:57.862275 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:57.865269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:57.870070 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e84ae193-9612-49d9-93e8-16fd04426420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f799b8e0fb4f44eea4230e4925f0ed2b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:57.870682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e84ae193-9612-49d9-93e8-16fd04426420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f799b8e0fb4f44eea4230e4925f0ed2b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:57.899691 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e84ae193-9612-49d9-93e8-16fd04426420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f799b8e0fb4f44eea4230e4925f0ed2b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:57.914088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e84ae193-9612-49d9-93e8-16fd04426420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f799b8e0fb4f44eea4230e4925f0ed2b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:57.915849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e84ae193-9612-49d9-93e8-16fd04426420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f799b8e0fb4f44eea4230e4925f0ed2b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:57.943975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1083/2184] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:57 2022] DELETE /identity/v3/users/f799b8e0fb4f44eea4230e4925f0ed2b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:57.945258 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f9b60-6cd3-4d37-bd17-82a05df00e0a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:57.948190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf23be-f35f-45e8-977f-81816ed38b99 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:57.948467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf23be-f35f-45e8-977f-81816ed38b99 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:57.948747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf23be-f35f-45e8-977f-81816ed38b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:57.996372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-89cf23be-f35f-45e8-977f-81816ed38b99 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:57.997621 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-89cf23be-f35f-45e8-977f-81816ed38b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:58.089378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680d80e-7df8-4add-962a-f1bd6e6f3f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:58.090697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680d80e-7df8-4add-962a-f1bd6e6f3f37 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.091162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680d80e-7df8-4add-962a-f1bd6e6f3f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.091951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680d80e-7df8-4add-962a-f1bd6e6f3f37 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.093081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680d80e-7df8-4add-962a-f1bd6e6f3f37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:58.095498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680d80e-7df8-4add-962a-f1bd6e6f3f37 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:58.116277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1084/2185] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:58.116277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2005687-c1a3-41be-8c35-50e99135fbd0 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.116277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2005687-c1a3-41be-8c35-50e99135fbd0 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.116277 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2005687-c1a3-41be-8c35-50e99135fbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.140083 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1102/2186] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:57 2022] POST /identity/v3/users => generated 287 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:58.141345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680d80e-7df8-4add-962a-f1bd6e6f3f37 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:58.145080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2680d80e-7df8-4add-962a-f1bd6e6f3f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:58.161600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f2005687-c1a3-41be-8c35-50e99135fbd0 None None] MFA Rules not processed for user `8fcf4142aa8340d692b1b3ed0aee7c87`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:58.163063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f2005687-c1a3-41be-8c35-50e99135fbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:58.250287 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680d80e-7df8-4add-962a-f1bd6e6f3f37 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:58.253505 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1085/2187] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:58.254558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2005687-c1a3-41be-8c35-50e99135fbd0 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:58.255944 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f2005687-c1a3-41be-8c35-50e99135fbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:58.260773 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa2ae5-8a9e-4321-972d-e6f41867367c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:58.261721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa2ae5-8a9e-4321-972d-e6f41867367c service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.261861 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa2ae5-8a9e-4321-972d-e6f41867367c service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.262101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa2ae5-8a9e-4321-972d-e6f41867367c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.270457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2005687-c1a3-41be-8c35-50e99135fbd0 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:58.273182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa2ae5-8a9e-4321-972d-e6f41867367c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:58.274661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaa2ae5-8a9e-4321-972d-e6f41867367c service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:58.276624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aab3b-df18-46f6-ac62-7ae3ac590cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:58.277471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aab3b-df18-46f6-ac62-7ae3ac590cca admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.277851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aab3b-df18-46f6-ac62-7ae3ac590cca admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.278202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911aab3b-df18-46f6-ac62-7ae3ac590cca admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.280382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911aab3b-df18-46f6-ac62-7ae3ac590cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:58.282114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911aab3b-df18-46f6-ac62-7ae3ac590cca admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:58.287019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1103/2188] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:47:58.288129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa2ae5-8a9e-4321-972d-e6f41867367c service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:58.289615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-eeaa2ae5-8a9e-4321-972d-e6f41867367c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:58.294147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1086/2189] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:58.298015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aab3b-df18-46f6-ac62-7ae3ac590cca admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:58.309469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911aab3b-df18-46f6-ac62-7ae3ac590cca admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:58.314545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6e7a8-76de-44ed-84f9-7843e3ea27bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:58.315296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6e7a8-76de-44ed-84f9-7843e3ea27bd admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.315644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6e7a8-76de-44ed-84f9-7843e3ea27bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.315997 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6e7a8-76de-44ed-84f9-7843e3ea27bd admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.316734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6e7a8-76de-44ed-84f9-7843e3ea27bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:58.318801 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6e7a8-76de-44ed-84f9-7843e3ea27bd admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:58.320342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaa2ae5-8a9e-4321-972d-e6f41867367c service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:58.330726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51855e14-6926-4446-8cb2-bb2219259d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:58.331353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51855e14-6926-4446-8cb2-bb2219259d6f admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.333289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51855e14-6926-4446-8cb2-bb2219259d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.333719 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51855e14-6926-4446-8cb2-bb2219259d6f admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.334364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51855e14-6926-4446-8cb2-bb2219259d6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:58.336231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1087/2190] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:58.337178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6e7a8-76de-44ed-84f9-7843e3ea27bd admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:58.338568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51855e14-6926-4446-8cb2-bb2219259d6f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:58.349726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6e7a8-76de-44ed-84f9-7843e3ea27bd admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:58.367438 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96553fea-3c67-44ee-bfec-d7e75b5c42ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:58.369606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96553fea-3c67-44ee-bfec-d7e75b5c42ab admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.369924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96553fea-3c67-44ee-bfec-d7e75b5c42ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.370241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96553fea-3c67-44ee-bfec-d7e75b5c42ab admin admin] PATH_INFO: `/v3/projects/47f943270eaa45a4b5aec895741d553d/users/efea00dacdca4d1d8e919fb446e164ea/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.391799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1104/2191] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:58 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:58.394950 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96553fea-3c67-44ee-bfec-d7e75b5c42ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f943270eaa45a4b5aec895741d553d, user_id=efea00dacdca4d1d8e919fb446e164ea, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:58.396254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51855e14-6926-4446-8cb2-bb2219259d6f admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:58.398427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96553fea-3c67-44ee-bfec-d7e75b5c42ab admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:58.417703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1088/2192] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:58 2022] PUT /identity/v3/projects/47f943270eaa45a4b5aec895741d553d/users/efea00dacdca4d1d8e919fb446e164ea/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:58.424411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70823602-3da3-4b23-90ae-47b119e816b2 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.425160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70823602-3da3-4b23-90ae-47b119e816b2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.425438 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70823602-3da3-4b23-90ae-47b119e816b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.442453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:58.457358 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-70823602-3da3-4b23-90ae-47b119e816b2 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:58.458978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-70823602-3da3-4b23-90ae-47b119e816b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:58.468440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:58.496768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51855e14-6926-4446-8cb2-bb2219259d6f admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:58.589190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1089/2193] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:58.596242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e6de4-0ea1-4ce9-b774-799f5f5c09e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:58.597772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e6de4-0ea1-4ce9-b774-799f5f5c09e7 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.598023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e6de4-0ea1-4ce9-b774-799f5f5c09e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.598407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e6de4-0ea1-4ce9-b774-799f5f5c09e7 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.599319 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e6de4-0ea1-4ce9-b774-799f5f5c09e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:58.601761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e6de4-0ea1-4ce9-b774-799f5f5c09e7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:58.624575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70823602-3da3-4b23-90ae-47b119e816b2 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:58.626656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-70823602-3da3-4b23-90ae-47b119e816b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:58.650160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1105/2194] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:58 2022] POST /identity/v3/users => generated 298 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:58.665054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e6de4-0ea1-4ce9-b774-799f5f5c09e7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:58.697367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e6de4-0ea1-4ce9-b774-799f5f5c09e7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:58.712541 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809415a-1f34-4419-8af0-a428ceec337e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:58.713349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809415a-1f34-4419-8af0-a428ceec337e admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.713482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809415a-1f34-4419-8af0-a428ceec337e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.718814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809415a-1f34-4419-8af0-a428ceec337e admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.719751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809415a-1f34-4419-8af0-a428ceec337e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:58.721746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809415a-1f34-4419-8af0-a428ceec337e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:58.734542 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1106/2195] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:47:58.739336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809415a-1f34-4419-8af0-a428ceec337e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:58.753815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809415a-1f34-4419-8af0-a428ceec337e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:58.758797 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70823602-3da3-4b23-90ae-47b119e816b2 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:58.760650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca81b81-a7a5-45a6-86d1-f882a40cc6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:58.762021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca81b81-a7a5-45a6-86d1-f882a40cc6d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.762413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca81b81-a7a5-45a6-86d1-f882a40cc6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.762815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca81b81-a7a5-45a6-86d1-f882a40cc6d7 admin admin] PATH_INFO: `/v3/projects/4c74255a5781460facaa444c24f2ef86/users/d33ac816c90d458788c5fa7ab2680bfb/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.768159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d573fd-87e9-4983-a906-163a12f7d059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:58.769240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d573fd-87e9-4983-a906-163a12f7d059 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.769240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d573fd-87e9-4983-a906-163a12f7d059 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.769804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d573fd-87e9-4983-a906-163a12f7d059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.784705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d573fd-87e9-4983-a906-163a12f7d059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:58.786237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d573fd-87e9-4983-a906-163a12f7d059 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:58.807970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1090/2196] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:47:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:47:58.811811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca81b81-a7a5-45a6-86d1-f882a40cc6d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c74255a5781460facaa444c24f2ef86, user_id=d33ac816c90d458788c5fa7ab2680bfb, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:58.827446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca81b81-a7a5-45a6-86d1-f882a40cc6d7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:58.831096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d573fd-87e9-4983-a906-163a12f7d059 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:58.834393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-54d573fd-87e9-4983-a906-163a12f7d059 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:58.854147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d573fd-87e9-4983-a906-163a12f7d059 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:58.860072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:58.861197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:58.861545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:58.861996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5 admin admin] PATH_INFO: `/v3/projects/34f73eeee4f145e7981f4b88a0fa23b8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:58.863566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f73eeee4f145e7981f4b88a0fa23b8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:58.866343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:58.874882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1107/2197] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:58 2022] PUT /identity/v3/projects/4c74255a5781460facaa444c24f2ef86/users/d33ac816c90d458788c5fa7ab2680bfb/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:58.879416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca81b81-a7a5-45a6-86d1-f882a40cc6d7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:58.956215 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f73eeee4f145e7981f4b88a0fa23b8', 'request_id': 'req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:58.967365 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f73eeee4f145e7981f4b88a0fa23b8', 'request_id': 'req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:58.970627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca81b81-a7a5-45a6-86d1-f882a40cc6d7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:58.976536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f73eeee4f145e7981f4b88a0fa23b8', 'request_id': 'req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:58.998118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5 admin admin] The token cache is being invalidate because project 34f73eeee4f145e7981f4b88a0fa23b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:47:58.998706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:59.000105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-47f70ef6-0a5b-41ca-9d4b-ded3b1d6b3e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:59.002200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1091/2198] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:47:58 2022] DELETE /identity/v3/projects/34f73eeee4f145e7981f4b88a0fa23b8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:59.007048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aab38-b816-4f8e-bfdb-bddaff925ebc None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.007400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aab38-b816-4f8e-bfdb-bddaff925ebc None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.007743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aab38-b816-4f8e-bfdb-bddaff925ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.051701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-579aab38-b816-4f8e-bfdb-bddaff925ebc None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:59.053043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-579aab38-b816-4f8e-bfdb-bddaff925ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:59.057154 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:59.074942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6b600-b857-48db-8ece-84334a6e9e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:59.075955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6b600-b857-48db-8ece-84334a6e9e21 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.076851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6b600-b857-48db-8ece-84334a6e9e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.077229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6b600-b857-48db-8ece-84334a6e9e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.078279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6b600-b857-48db-8ece-84334a6e9e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:59.080270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6b600-b857-48db-8ece-84334a6e9e21 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:59.098500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1108/2199] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:58 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:59.105300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6b600-b857-48db-8ece-84334a6e9e21 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:59.107803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b0f6b600-b857-48db-8ece-84334a6e9e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:59.128090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:59.142373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6b600-b857-48db-8ece-84334a6e9e21 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:59.165037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f70c3-0baa-40ad-ada1-33bdfcfd1dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:59.168113 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f70c3-0baa-40ad-ada1-33bdfcfd1dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.168868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f70c3-0baa-40ad-ada1-33bdfcfd1dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.169587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f70c3-0baa-40ad-ada1-33bdfcfd1dc1 admin admin] PATH_INFO: `/v3/projects/4c74255a5781460facaa444c24f2ef86/users/d33ac816c90d458788c5fa7ab2680bfb/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.174803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f70c3-0baa-40ad-ada1-33bdfcfd1dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c74255a5781460facaa444c24f2ef86, user_id=d33ac816c90d458788c5fa7ab2680bfb, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:59.177635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f70c3-0baa-40ad-ada1-33bdfcfd1dc1 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:59.205607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1109/2200] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:59 2022] PUT /identity/v3/projects/4c74255a5781460facaa444c24f2ef86/users/d33ac816c90d458788c5fa7ab2680bfb/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:59.206970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f70c3-0baa-40ad-ada1-33bdfcfd1dc1 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:59.210723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ef5f70c3-0baa-40ad-ada1-33bdfcfd1dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:59.212097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1092/2201] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:47:59.226238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb528cb3-3ab1-4d5e-948a-1c4a075986fd None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.228346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb528cb3-3ab1-4d5e-948a-1c4a075986fd None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.228723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb528cb3-3ab1-4d5e-948a-1c4a075986fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.304402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-eb528cb3-3ab1-4d5e-948a-1c4a075986fd None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:47:59.305732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-eb528cb3-3ab1-4d5e-948a-1c4a075986fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:59.385800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f70c3-0baa-40ad-ada1-33bdfcfd1dc1 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:59.396819 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:59.398369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.399369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.400153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] PATH_INFO: `/v3/users/e091246128db4727ba507029b25d3774` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.402346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e091246128db4727ba507029b25d3774)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:59.405322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:59.413571 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1093/2202] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:47:59.417958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb528cb3-3ab1-4d5e-948a-1c4a075986fd None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:59.434076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb528cb3-3ab1-4d5e-948a-1c4a075986fd None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:59.448171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffd691-aa4c-4df4-980b-9c006c0a23eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:59.448805 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffd691-aa4c-4df4-980b-9c006c0a23eb admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.449566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffd691-aa4c-4df4-980b-9c006c0a23eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.449984 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffd691-aa4c-4df4-980b-9c006c0a23eb admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.450610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ffd691-aa4c-4df4-980b-9c006c0a23eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:47:59.450913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ffd691-aa4c-4df4-980b-9c006c0a23eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:59.453753 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ffd691-aa4c-4df4-980b-9c006c0a23eb admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:59.479512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1094/2203] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:47:59.483786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffd691-aa4c-4df4-980b-9c006c0a23eb admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:59.495203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffd691-aa4c-4df4-980b-9c006c0a23eb admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:59.500924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1645ac9-09d8-4da5-a309-9195790619c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:59.501564 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1645ac9-09d8-4da5-a309-9195790619c1 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.501878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1645ac9-09d8-4da5-a309-9195790619c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.502159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1645ac9-09d8-4da5-a309-9195790619c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.502589 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e091246128db4727ba507029b25d3774'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:59.503793 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1645ac9-09d8-4da5-a309-9195790619c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:59.504979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e091246128db4727ba507029b25d3774'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:59.505434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1645ac9-09d8-4da5-a309-9195790619c1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:59.521483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e091246128db4727ba507029b25d3774'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:59.522586 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e091246128db4727ba507029b25d3774'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:59.545483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e091246128db4727ba507029b25d3774'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:47:59.561358 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1095/2204] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:47:59 2022] POST /identity/v3/projects => generated 369 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:59.566992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1645ac9-09d8-4da5-a309-9195790619c1 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:59.568976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b1645ac9-09d8-4da5-a309-9195790619c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:47:59.577619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1110/2205] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:47:59 2022] DELETE /identity/v3/users/e091246128db4727ba507029b25d3774 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:47:59.688736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1645ac9-09d8-4da5-a309-9195790619c1 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:59.810235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:47:59.832041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b934bf-fe7d-44a5-ae46-dbf85ae3d45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:59.832667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b934bf-fe7d-44a5-ae46-dbf85ae3d45c admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.832951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b934bf-fe7d-44a5-ae46-dbf85ae3d45c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.833220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b934bf-fe7d-44a5-ae46-dbf85ae3d45c admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.833907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b934bf-fe7d-44a5-ae46-dbf85ae3d45c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:59.835971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b934bf-fe7d-44a5-ae46-dbf85ae3d45c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:59.874346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1096/2206] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:47:59 2022] POST /identity/v3/users => generated 285 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:47:59.875448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:59.889459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46418a0-dde4-46e7-98a6-fceb121d3676 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:59.898548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b4f08-e8e9-4977-af3c-2b5f98b15479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:59.900591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b4f08-e8e9-4977-af3c-2b5f98b15479 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.901045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b4f08-e8e9-4977-af3c-2b5f98b15479 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.901338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b4f08-e8e9-4977-af3c-2b5f98b15479 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.902466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b4f08-e8e9-4977-af3c-2b5f98b15479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:59.905399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b4f08-e8e9-4977-af3c-2b5f98b15479 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:59.925978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1111/2207] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:59.929112 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b934bf-fe7d-44a5-ae46-dbf85ae3d45c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:47:59.943354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b934bf-fe7d-44a5-ae46-dbf85ae3d45c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:47:59.948175 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866541d-70a1-4c49-a938-190dd0b4eb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:47:59.948804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866541d-70a1-4c49-a938-190dd0b4eb13 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:47:59.949093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866541d-70a1-4c49-a938-190dd0b4eb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:47:59.949357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866541d-70a1-4c49-a938-190dd0b4eb13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:47:59.950318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866541d-70a1-4c49-a938-190dd0b4eb13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:47:59.953930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866541d-70a1-4c49-a938-190dd0b4eb13 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:47:59.976675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1097/2208] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:47:59.978131 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b4f08-e8e9-4977-af3c-2b5f98b15479 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:00.004240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b4f08-e8e9-4977-af3c-2b5f98b15479 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:00.010909 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274e4bc-a397-41bf-a9dd-b446d65bdedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:00.013281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274e4bc-a397-41bf-a9dd-b446d65bdedd admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:00.013679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274e4bc-a397-41bf-a9dd-b446d65bdedd admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:00.014400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274e4bc-a397-41bf-a9dd-b446d65bdedd admin admin] PATH_INFO: `/v3/projects/bf80faf02bc24c388aa84f92f64681e5/users/18bce21cd99842e8abef570dce4fcdb3/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:00.040617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274e4bc-a397-41bf-a9dd-b446d65bdedd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf80faf02bc24c388aa84f92f64681e5, user_id=18bce21cd99842e8abef570dce4fcdb3, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:00.043417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274e4bc-a397-41bf-a9dd-b446d65bdedd admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:00.067241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1112/2209] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:47:59 2022] PUT /identity/v3/projects/bf80faf02bc24c388aa84f92f64681e5/users/18bce21cd99842e8abef570dce4fcdb3/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:00.080402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b89d99-da2e-4c13-95e6-6ec2ede29d2d None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:00.084287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b89d99-da2e-4c13-95e6-6ec2ede29d2d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:00.084287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b89d99-da2e-4c13-95e6-6ec2ede29d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:00.113825 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274e4bc-a397-41bf-a9dd-b446d65bdedd admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:00.115631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c274e4bc-a397-41bf-a9dd-b446d65bdedd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:00.150809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:00.179452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c2b89d99-da2e-4c13-95e6-6ec2ede29d2d None None] MFA Rules not processed for user `18bce21cd99842e8abef570dce4fcdb3`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:00.180728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c2b89d99-da2e-4c13-95e6-6ec2ede29d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:00.259999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274e4bc-a397-41bf-a9dd-b446d65bdedd admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:00.272005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:00.273119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:00.273418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:00.273724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] PATH_INFO: `/v3/projects/738514e187e44365980b7794bc9923f2` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:00.275391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738514e187e44365980b7794bc9923f2)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:00.278613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:00.360884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1098/2210] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:48:00.383299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738514e187e44365980b7794bc9923f2', 'request_id': 'req-d1a17255-6f8e-4a53-9843-64af5aacfd12'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:00.387276 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b89d99-da2e-4c13-95e6-6ec2ede29d2d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:00.389791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c2b89d99-da2e-4c13-95e6-6ec2ede29d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:00.390975 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738514e187e44365980b7794bc9923f2', 'request_id': 'req-d1a17255-6f8e-4a53-9843-64af5aacfd12'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:00.401791 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738514e187e44365980b7794bc9923f2', 'request_id': 'req-d1a17255-6f8e-4a53-9843-64af5aacfd12'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:00.438078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] The token cache is being invalidate because project 738514e187e44365980b7794bc9923f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:48:00.438713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:00.442137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:00.444438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1113/2211] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:48:00 2022] DELETE /identity/v3/projects/738514e187e44365980b7794bc9923f2 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:00.474740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:00.480025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:00.500963 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b89d99-da2e-4c13-95e6-6ec2ede29d2d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:00.598360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7dd211-d4d8-4756-91cf-8a22f4467a8e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '913a866ddc074b7f9aa142c70da98a69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:00.600016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dd211-d4d8-4756-91cf-8a22f4467a8e service glance] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:00.600016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dd211-d4d8-4756-91cf-8a22f4467a8e service glance] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:00.600016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dd211-d4d8-4756-91cf-8a22f4467a8e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:00.602867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2b7dd211-d4d8-4756-91cf-8a22f4467a8e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:00.640952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:00.682947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a17255-6f8e-4a53-9843-64af5aacfd12 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:00.688600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:00.690143 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:00.690923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:00.691554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] PATH_INFO: `/v3/users/2e3a76b0ef1744bbb1b877ef0b40b80b` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:00.692806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e3a76b0ef1744bbb1b877ef0b40b80b)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:00.694834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:00.756206 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7dd211-d4d8-4756-91cf-8a22f4467a8e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:00.767279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7dd211-d4d8-4756-91cf-8a22f4467a8e service glance] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:00.767600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:00.771278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e3a76b0ef1744bbb1b877ef0b40b80b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:00.772210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e3a76b0ef1744bbb1b877ef0b40b80b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:00.785648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e3a76b0ef1744bbb1b877ef0b40b80b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:00.786533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e3a76b0ef1744bbb1b877ef0b40b80b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:00.797831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1099/2212] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:48:00.803179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e07b6-058d-492a-a669-0535dd23eb0d None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:00.807087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e07b6-058d-492a-a669-0535dd23eb0d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:00.807896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e3a76b0ef1744bbb1b877ef0b40b80b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:00.808752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e07b6-058d-492a-a669-0535dd23eb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:00.840972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1114/2213] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:48:00 2022] DELETE /identity/v3/users/2e3a76b0ef1744bbb1b877ef0b40b80b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:00.884722 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-ac1e07b6-058d-492a-a669-0535dd23eb0d None None] MFA Rules not processed for user `efea00dacdca4d1d8e919fb446e164ea`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:00.885955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ac1e07b6-058d-492a-a669-0535dd23eb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:00.995661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1100/2214] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:01.022799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:01.025203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:01.110383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fecc57-b8c9-4108-8f42-89469218e431 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:01.114253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5f706-00c8-4e0a-bcc8-606fcdb4f845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:01.114872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5f706-00c8-4e0a-bcc8-606fcdb4f845 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:01.115194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5f706-00c8-4e0a-bcc8-606fcdb4f845 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:01.115536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5f706-00c8-4e0a-bcc8-606fcdb4f845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:01.125344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5f706-00c8-4e0a-bcc8-606fcdb4f845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:01.126898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5f706-00c8-4e0a-bcc8-606fcdb4f845 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:01.140139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1115/2215] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:48:01.351036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e07b6-058d-492a-a669-0535dd23eb0d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:01.353093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ac1e07b6-058d-492a-a669-0535dd23eb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:01.569973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e07b6-058d-492a-a669-0535dd23eb0d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:01.590348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bd695-37ad-4e50-a2a0-66eaa7c13fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:01.592240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bd695-37ad-4e50-a2a0-66eaa7c13fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:01.594924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bd695-37ad-4e50-a2a0-66eaa7c13fad admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:01.596408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bd695-37ad-4e50-a2a0-66eaa7c13fad admin admin] PATH_INFO: `/v3/projects/e94026aaac6a4201aace07d8dec67117` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:01.599229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bd695-37ad-4e50-a2a0-66eaa7c13fad admin admin] RBAC: Authorizing `identity:delete_project(project_id=e94026aaac6a4201aace07d8dec67117)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:01.605285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bd695-37ad-4e50-a2a0-66eaa7c13fad admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:01.730173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-890bd695-37ad-4e50-a2a0-66eaa7c13fad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e94026aaac6a4201aace07d8dec67117', 'request_id': 'req-890bd695-37ad-4e50-a2a0-66eaa7c13fad'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:01.741779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-890bd695-37ad-4e50-a2a0-66eaa7c13fad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e94026aaac6a4201aace07d8dec67117', 'request_id': 'req-890bd695-37ad-4e50-a2a0-66eaa7c13fad'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:01.748090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:01.760662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-890bd695-37ad-4e50-a2a0-66eaa7c13fad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e94026aaac6a4201aace07d8dec67117', 'request_id': 'req-890bd695-37ad-4e50-a2a0-66eaa7c13fad'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:01.784636 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-890bd695-37ad-4e50-a2a0-66eaa7c13fad admin admin] The token cache is being invalidate because project e94026aaac6a4201aace07d8dec67117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:48:01.785599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-890bd695-37ad-4e50-a2a0-66eaa7c13fad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:01.786691 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-890bd695-37ad-4e50-a2a0-66eaa7c13fad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:01.789504 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1101/2216] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:48:01 2022] DELETE /identity/v3/projects/e94026aaac6a4201aace07d8dec67117 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:48:01.795584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6905a2-e25c-4a4c-a841-a51448c9285b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:01.795584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6905a2-e25c-4a4c-a841-a51448c9285b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:01.810407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6905a2-e25c-4a4c-a841-a51448c9285b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:01.842974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-9b6905a2-e25c-4a4c-a841-a51448c9285b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:01.857108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9b6905a2-e25c-4a4c-a841-a51448c9285b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:02.038468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1116/2217] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:48:02.046715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e727f5-bb5b-4e91-9ddf-6b17dc3c2465 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:02.047351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e727f5-bb5b-4e91-9ddf-6b17dc3c2465 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:02.047351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e727f5-bb5b-4e91-9ddf-6b17dc3c2465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:02.081501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-e8e727f5-bb5b-4e91-9ddf-6b17dc3c2465 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:02.083641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e8e727f5-bb5b-4e91-9ddf-6b17dc3c2465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:02.100440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1102/2218] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:02.104801 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6905a2-e25c-4a4c-a841-a51448c9285b None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:02.121664 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6905a2-e25c-4a4c-a841-a51448c9285b None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:02.134300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fbfe7-9d5d-43d2-bbea-da8023d69316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:02.134937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fbfe7-9d5d-43d2-bbea-da8023d69316 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:02.135215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fbfe7-9d5d-43d2-bbea-da8023d69316 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:02.135527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fbfe7-9d5d-43d2-bbea-da8023d69316 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:02.136185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fbfe7-9d5d-43d2-bbea-da8023d69316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:48:02.136493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fbfe7-9d5d-43d2-bbea-da8023d69316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:02.138157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fbfe7-9d5d-43d2-bbea-da8023d69316 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:02.167953 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1117/2219] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:48:02.174106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e727f5-bb5b-4e91-9ddf-6b17dc3c2465 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:02.185922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e727f5-bb5b-4e91-9ddf-6b17dc3c2465 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:02.204219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01199cb8-77b6-463e-873a-684c3aa88407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:02.206420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01199cb8-77b6-463e-873a-684c3aa88407 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:02.206881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01199cb8-77b6-463e-873a-684c3aa88407 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:02.207298 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01199cb8-77b6-463e-873a-684c3aa88407 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:02.208143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01199cb8-77b6-463e-873a-684c3aa88407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:02.211364 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01199cb8-77b6-463e-873a-684c3aa88407 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:02.254920 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1103/2220] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:48:02 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:48:02.255944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fbfe7-9d5d-43d2-bbea-da8023d69316 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:02.273866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fbfe7-9d5d-43d2-bbea-da8023d69316 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:02.363980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:02.381302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a19b-be92-47be-b86c-7261feda2783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:02.382184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a19b-be92-47be-b86c-7261feda2783 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:02.382654 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a19b-be92-47be-b86c-7261feda2783 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:02.383660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4a19b-be92-47be-b86c-7261feda2783 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:02.384156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4a19b-be92-47be-b86c-7261feda2783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:02.386383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4a19b-be92-47be-b86c-7261feda2783 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:02.417181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01199cb8-77b6-463e-873a-684c3aa88407 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:02.427890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1118/2221] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:48:02 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:48:02.430794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01199cb8-77b6-463e-873a-684c3aa88407 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:02.438129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f7b4a-cdc5-43a6-91e8-f144ea712429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:02.439236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f7b4a-cdc5-43a6-91e8-f144ea712429 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:02.439803 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f7b4a-cdc5-43a6-91e8-f144ea712429 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:02.440966 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f7b4a-cdc5-43a6-91e8-f144ea712429 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:02.441933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f7b4a-cdc5-43a6-91e8-f144ea712429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:02.443827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f7b4a-cdc5-43a6-91e8-f144ea712429 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:02.463680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a19b-be92-47be-b86c-7261feda2783 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:02.472438 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1104/2222] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:48:02.477647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4a19b-be92-47be-b86c-7261feda2783 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:02.483200 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fe29e-2dc5-4896-bd6b-ee20dc92b737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:02.484011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fe29e-2dc5-4896-bd6b-ee20dc92b737 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:02.484512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fe29e-2dc5-4896-bd6b-ee20dc92b737 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:02.484890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fe29e-2dc5-4896-bd6b-ee20dc92b737 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:02.485782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fe29e-2dc5-4896-bd6b-ee20dc92b737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:02.488706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fe29e-2dc5-4896-bd6b-ee20dc92b737 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:02.508660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f7b4a-cdc5-43a6-91e8-f144ea712429 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:02.509522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1119/2223] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:48:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:48:02.520590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f7b4a-cdc5-43a6-91e8-f144ea712429 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:02.527339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd1e07-8312-420b-a8b6-6a5d027775a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:02.528121 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd1e07-8312-420b-a8b6-6a5d027775a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:02.528503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd1e07-8312-420b-a8b6-6a5d027775a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:02.528892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd1e07-8312-420b-a8b6-6a5d027775a6 admin admin] PATH_INFO: `/v3/projects/2079f15807f04db1a080d438d12fc5fa/users/f537a87cf752463da19bde110e86dd9a/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:02.557340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd1e07-8312-420b-a8b6-6a5d027775a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2079f15807f04db1a080d438d12fc5fa, user_id=f537a87cf752463da19bde110e86dd9a, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:02.561033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd1e07-8312-420b-a8b6-6a5d027775a6 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:02.588789 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1105/2224] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:48:02 2022] PUT /identity/v3/projects/2079f15807f04db1a080d438d12fc5fa/users/f537a87cf752463da19bde110e86dd9a/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:02.600400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad26fce-dd08-4e2e-8c77-cdc5b8ffca97 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:02.600774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad26fce-dd08-4e2e-8c77-cdc5b8ffca97 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:02.601171 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad26fce-dd08-4e2e-8c77-cdc5b8ffca97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:02.638037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6ad26fce-dd08-4e2e-8c77-cdc5b8ffca97 None None] MFA Rules not processed for user `f537a87cf752463da19bde110e86dd9a`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:02.639479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6ad26fce-dd08-4e2e-8c77-cdc5b8ffca97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:02.837118 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd1e07-8312-420b-a8b6-6a5d027775a6 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:02.844292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1120/2225] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:02.950479 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd1e07-8312-420b-a8b6-6a5d027775a6 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:02.958451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb661c0-2d09-43aa-bdb7-32043c2e8d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:02.959363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb661c0-2d09-43aa-bdb7-32043c2e8d13 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:02.959766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb661c0-2d09-43aa-bdb7-32043c2e8d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:02.959922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb661c0-2d09-43aa-bdb7-32043c2e8d13 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:02.960811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb661c0-2d09-43aa-bdb7-32043c2e8d13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:02.962724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb661c0-2d09-43aa-bdb7-32043c2e8d13 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:02.994297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1106/2226] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:48:02 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:48:02.997641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad26fce-dd08-4e2e-8c77-cdc5b8ffca97 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:03.010239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad26fce-dd08-4e2e-8c77-cdc5b8ffca97 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:03.079686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3dbda-1e63-4d3a-9eaf-e883d84bc20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:03.080402 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3dbda-1e63-4d3a-9eaf-e883d84bc20f admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:03.080777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3dbda-1e63-4d3a-9eaf-e883d84bc20f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:03.081130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3dbda-1e63-4d3a-9eaf-e883d84bc20f admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:03.081873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3dbda-1e63-4d3a-9eaf-e883d84bc20f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:03.083935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3dbda-1e63-4d3a-9eaf-e883d84bc20f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:03.107428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1121/2227] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:48:02 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:48:03.111308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb661c0-2d09-43aa-bdb7-32043c2e8d13 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:03.124602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb661c0-2d09-43aa-bdb7-32043c2e8d13 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:03.134718 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c7bd6-7401-4543-b595-910114820a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:03.135485 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c7bd6-7401-4543-b595-910114820a87 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:03.135868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c7bd6-7401-4543-b595-910114820a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:03.136209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c7bd6-7401-4543-b595-910114820a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:03.137614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c7bd6-7401-4543-b595-910114820a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:03.139292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c7bd6-7401-4543-b595-910114820a87 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:03.161422 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1107/2228] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:48:03.163469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3dbda-1e63-4d3a-9eaf-e883d84bc20f admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:03.177641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3dbda-1e63-4d3a-9eaf-e883d84bc20f admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:03.183619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b93f68-b303-4cdb-9ef3-1ede6b205819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:03.184561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b93f68-b303-4cdb-9ef3-1ede6b205819 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:03.184923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b93f68-b303-4cdb-9ef3-1ede6b205819 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:03.185380 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b93f68-b303-4cdb-9ef3-1ede6b205819 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:03.186289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b93f68-b303-4cdb-9ef3-1ede6b205819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:03.188523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b93f68-b303-4cdb-9ef3-1ede6b205819 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:03.209225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1122/2229] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:48:03.213772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c7bd6-7401-4543-b595-910114820a87 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:03.225901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c7bd6-7401-4543-b595-910114820a87 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:03.231326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a36bc-3bea-4e8b-9cc9-da39ee2626dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:03.232032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a36bc-3bea-4e8b-9cc9-da39ee2626dd admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:03.232315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a36bc-3bea-4e8b-9cc9-da39ee2626dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:03.232639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a36bc-3bea-4e8b-9cc9-da39ee2626dd admin admin] PATH_INFO: `/v3/projects/877556067d064c8c974c29b7fc7e0c81/users/cd1d6f7834e14e1daa1e4aa5f8f4525b/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:03.249920 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a36bc-3bea-4e8b-9cc9-da39ee2626dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=877556067d064c8c974c29b7fc7e0c81, user_id=cd1d6f7834e14e1daa1e4aa5f8f4525b, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:03.252688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a36bc-3bea-4e8b-9cc9-da39ee2626dd admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:03.274483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1108/2230] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:48:03 2022] PUT /identity/v3/projects/877556067d064c8c974c29b7fc7e0c81/users/cd1d6f7834e14e1daa1e4aa5f8f4525b/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:03.287475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88595c3a-2b04-4c8a-9b71-403cfeb35a74 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:03.287850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88595c3a-2b04-4c8a-9b71-403cfeb35a74 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:03.288138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88595c3a-2b04-4c8a-9b71-403cfeb35a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:03.327039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-88595c3a-2b04-4c8a-9b71-403cfeb35a74 None None] MFA Rules not processed for user `cd1d6f7834e14e1daa1e4aa5f8f4525b`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:03.329717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-88595c3a-2b04-4c8a-9b71-403cfeb35a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:03.384540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:03.465468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1123/2231] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:03.466984 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a36bc-3bea-4e8b-9cc9-da39ee2626dd admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:03.535887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:03.603201 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a36bc-3bea-4e8b-9cc9-da39ee2626dd admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:03.608792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fac84-e2be-4d5c-a0b5-8f451a830368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:03.609420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fac84-e2be-4d5c-a0b5-8f451a830368 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:03.609713 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fac84-e2be-4d5c-a0b5-8f451a830368 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:03.610165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fac84-e2be-4d5c-a0b5-8f451a830368 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:03.610817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fac84-e2be-4d5c-a0b5-8f451a830368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:03.612992 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fac84-e2be-4d5c-a0b5-8f451a830368 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:03.675561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1109/2232] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:48:03 2022] POST /identity/v3/projects => generated 375 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:48:03.682892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88595c3a-2b04-4c8a-9b71-403cfeb35a74 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:03.714447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88595c3a-2b04-4c8a-9b71-403cfeb35a74 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:03.837710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f616e-c60f-475d-9f63-957eed0e6015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:03.838345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f616e-c60f-475d-9f63-957eed0e6015 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:03.838623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f616e-c60f-475d-9f63-957eed0e6015 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:03.838892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f616e-c60f-475d-9f63-957eed0e6015 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:03.839527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f616e-c60f-475d-9f63-957eed0e6015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:03.841650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f616e-c60f-475d-9f63-957eed0e6015 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:03.890689 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1124/2233] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:48:03 2022] POST /identity/v3/users => generated 294 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:48:03.893576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fac84-e2be-4d5c-a0b5-8f451a830368 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:03.920858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fac84-e2be-4d5c-a0b5-8f451a830368 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:03.951199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf26d4-0d51-4b01-bc56-b7185e47be73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:03.952101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf26d4-0d51-4b01-bc56-b7185e47be73 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:03.952441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf26d4-0d51-4b01-bc56-b7185e47be73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:03.952723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf26d4-0d51-4b01-bc56-b7185e47be73 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:03.953397 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf26d4-0d51-4b01-bc56-b7185e47be73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:03.959317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf26d4-0d51-4b01-bc56-b7185e47be73 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:03.989473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1110/2234] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:48:03.993778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f616e-c60f-475d-9f63-957eed0e6015 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:04.013042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cbfa7-3b98-4987-9551-c5971346df12 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:04.013042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cbfa7-3b98-4987-9551-c5971346df12 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:04.013463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299cbfa7-3b98-4987-9551-c5971346df12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:04.017227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f616e-c60f-475d-9f63-957eed0e6015 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:04.022203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c64aa-85ba-4c75-a74f-8d2c3cf565f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:04.022917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c64aa-85ba-4c75-a74f-8d2c3cf565f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:04.023198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c64aa-85ba-4c75-a74f-8d2c3cf565f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:04.023481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668c64aa-85ba-4c75-a74f-8d2c3cf565f6 admin admin] PATH_INFO: `/v3/projects/12037595ad034bcab1a7e371c8cce028/users/1208d0817edb4fa6ab8b14fd30c27bde/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:04.045610 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c64aa-85ba-4c75-a74f-8d2c3cf565f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12037595ad034bcab1a7e371c8cce028, user_id=1208d0817edb4fa6ab8b14fd30c27bde, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:04.049810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668c64aa-85ba-4c75-a74f-8d2c3cf565f6 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:04.055070 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-299cbfa7-3b98-4987-9551-c5971346df12 None None] MFA Rules not processed for user `efea00dacdca4d1d8e919fb446e164ea`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:04.056423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-299cbfa7-3b98-4987-9551-c5971346df12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:04.082726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1125/2235] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:48:03 2022] PUT /identity/v3/projects/12037595ad034bcab1a7e371c8cce028/users/1208d0817edb4fa6ab8b14fd30c27bde/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:04.082726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c64aa-85ba-4c75-a74f-8d2c3cf565f6 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:04.210616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668c64aa-85ba-4c75-a74f-8d2c3cf565f6 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:04.235582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e13300-8fe0-4c44-8e7b-ff5f6c495dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:04.236282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e13300-8fe0-4c44-8e7b-ff5f6c495dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:04.236645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e13300-8fe0-4c44-8e7b-ff5f6c495dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:04.237634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e13300-8fe0-4c44-8e7b-ff5f6c495dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:04.238383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e13300-8fe0-4c44-8e7b-ff5f6c495dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:04.240080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e13300-8fe0-4c44-8e7b-ff5f6c495dd3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:04.249192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1111/2236] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:48:04.259015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1126/2237] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:48:04.272417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cbfa7-3b98-4987-9551-c5971346df12 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:04.302405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299cbfa7-3b98-4987-9551-c5971346df12 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:04.308199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5169a-b36d-4d4e-a733-51b462886984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:04.308865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5169a-b36d-4d4e-a733-51b462886984 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:04.309154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5169a-b36d-4d4e-a733-51b462886984 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:04.309429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5169a-b36d-4d4e-a733-51b462886984 admin admin] PATH_INFO: `/v3/projects/12037595ad034bcab1a7e371c8cce028/users/1208d0817edb4fa6ab8b14fd30c27bde/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:04.311698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5169a-b36d-4d4e-a733-51b462886984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12037595ad034bcab1a7e371c8cce028, user_id=1208d0817edb4fa6ab8b14fd30c27bde, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:04.315926 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5169a-b36d-4d4e-a733-51b462886984 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:04.342201 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1112/2238] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:48:04 2022] PUT /identity/v3/projects/12037595ad034bcab1a7e371c8cce028/users/1208d0817edb4fa6ab8b14fd30c27bde/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:04.347684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56387f-8396-40e7-a4e2-22d7b2d09657 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:04.348063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56387f-8396-40e7-a4e2-22d7b2d09657 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:04.348243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56387f-8396-40e7-a4e2-22d7b2d09657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:04.385293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-ca56387f-8396-40e7-a4e2-22d7b2d09657 None None] MFA Rules not processed for user `1208d0817edb4fa6ab8b14fd30c27bde`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:04.386833 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ca56387f-8396-40e7-a4e2-22d7b2d09657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:04.413352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:04.498211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1127/2239] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:04.508158 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5169a-b36d-4d4e-a733-51b462886984 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:04.509989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7ef5169a-b36d-4d4e-a733-51b462886984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:04.619877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5169a-b36d-4d4e-a733-51b462886984 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:04.627577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce136f0-04ba-453f-81ee-ffa3734636e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:04.629365 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce136f0-04ba-453f-81ee-ffa3734636e2 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:04.629952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce136f0-04ba-453f-81ee-ffa3734636e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:04.630234 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce136f0-04ba-453f-81ee-ffa3734636e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:04.726766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:04.744752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce136f0-04ba-453f-81ee-ffa3734636e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:04.746817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce136f0-04ba-453f-81ee-ffa3734636e2 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:04.766995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1113/2240] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:48:06.488083 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb59602-ed93-4c6a-8460-410b10086de9 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:06.488623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb59602-ed93-4c6a-8460-410b10086de9 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:06.488623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb59602-ed93-4c6a-8460-410b10086de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:06.521598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-9eb59602-ed93-4c6a-8460-410b10086de9 None None] MFA Rules not processed for user `d33ac816c90d458788c5fa7ab2680bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:06.523032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9eb59602-ed93-4c6a-8460-410b10086de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:06.637157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1128/2241] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:06.649791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce136f0-04ba-453f-81ee-ffa3734636e2 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:06.651183 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4ce136f0-04ba-453f-81ee-ffa3734636e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:06.729428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce136f0-04ba-453f-81ee-ffa3734636e2 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:06.736479 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef67f39-51f5-423d-9be5-e51998023831 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '913a866ddc074b7f9aa142c70da98a69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:06.737777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef67f39-51f5-423d-9be5-e51998023831 service glance] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:06.738122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef67f39-51f5-423d-9be5-e51998023831 service glance] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:06.738330 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef67f39-51f5-423d-9be5-e51998023831 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:06.740285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0ef67f39-51f5-423d-9be5-e51998023831 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:06.826230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef67f39-51f5-423d-9be5-e51998023831 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:06.839298 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef67f39-51f5-423d-9be5-e51998023831 service glance] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:06.862199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1114/2242] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:48:07.054924 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61e0de-9075-4bee-8ec2-85417b6488d2 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:07.055409 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61e0de-9075-4bee-8ec2-85417b6488d2 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:07.056032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61e0de-9075-4bee-8ec2-85417b6488d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:07.085047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-ca61e0de-9075-4bee-8ec2-85417b6488d2 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:07.086587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ca61e0de-9075-4bee-8ec2-85417b6488d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:07.200044 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1129/2243] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:48:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:07.269584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef67f39-51f5-423d-9be5-e51998023831 service glance] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:07.272378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0ef67f39-51f5-423d-9be5-e51998023831 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:07.383795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef67f39-51f5-423d-9be5-e51998023831 service glance] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:07.401016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8c0a8-5bc3-40bd-a3f2-f43c4621cf32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:07.403311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8c0a8-5bc3-40bd-a3f2-f43c4621cf32 service swift] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:07.403605 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8c0a8-5bc3-40bd-a3f2-f43c4621cf32 service swift] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:07.403885 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8c0a8-5bc3-40bd-a3f2-f43c4621cf32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:07.414167 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8c0a8-5bc3-40bd-a3f2-f43c4621cf32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:07.415860 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8c0a8-5bc3-40bd-a3f2-f43c4621cf32 service swift] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:07.426778 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1115/2244] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:48:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:48:07.603538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61e0de-9075-4bee-8ec2-85417b6488d2 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:07.614860 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61e0de-9075-4bee-8ec2-85417b6488d2 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:07.634712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177ee80-b28a-4fb4-92f3-f3012fc4f30a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:07.635363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177ee80-b28a-4fb4-92f3-f3012fc4f30a service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:07.635652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177ee80-b28a-4fb4-92f3-f3012fc4f30a service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:07.636061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177ee80-b28a-4fb4-92f3-f3012fc4f30a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:07.727164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:07.760986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177ee80-b28a-4fb4-92f3-f3012fc4f30a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:07.763078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177ee80-b28a-4fb4-92f3-f3012fc4f30a service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:07.794495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1130/2245] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:48:08.770138 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bdd74-1125-4a71-a22e-79755332dfae None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:08.770640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bdd74-1125-4a71-a22e-79755332dfae None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:08.770913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bdd74-1125-4a71-a22e-79755332dfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:08.795810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-076bdd74-1125-4a71-a22e-79755332dfae None None] MFA Rules not processed for user `acf57eb971eb4f78b173a5ab47160bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:08.811241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-076bdd74-1125-4a71-a22e-79755332dfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:08.850994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177ee80-b28a-4fb4-92f3-f3012fc4f30a service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:08.871100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177ee80-b28a-4fb4-92f3-f3012fc4f30a service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:08.879019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269827be-8e74-4369-b058-9efd2b14112b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:08.879734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269827be-8e74-4369-b058-9efd2b14112b service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:08.880076 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269827be-8e74-4369-b058-9efd2b14112b service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:08.880561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269827be-8e74-4369-b058-9efd2b14112b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:08.936821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269827be-8e74-4369-b058-9efd2b14112b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:08.936821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269827be-8e74-4369-b058-9efd2b14112b service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:08.937149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:08.966863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1131/2246] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:48:08.990849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1116/2247] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:14.375672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269827be-8e74-4369-b058-9efd2b14112b service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:14.457709 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269827be-8e74-4369-b058-9efd2b14112b service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:14.463053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:14.464075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:14.464524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:14.464994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] PATH_INFO: `/v3/users/f537a87cf752463da19bde110e86dd9a` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:14.466308 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f537a87cf752463da19bde110e86dd9a)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:14.468678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:14.536299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f537a87cf752463da19bde110e86dd9a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:14.537886 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f537a87cf752463da19bde110e86dd9a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:14.552042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f537a87cf752463da19bde110e86dd9a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:14.553140 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f537a87cf752463da19bde110e86dd9a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:14.568676 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f537a87cf752463da19bde110e86dd9a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:14.606559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1132/2248] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:48:14 2022] DELETE /identity/v3/users/f537a87cf752463da19bde110e86dd9a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:14.611067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32886e5b-a2a2-4270-b548-bf631d056fb8 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:14.612920 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32886e5b-a2a2-4270-b548-bf631d056fb8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:14.613679 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32886e5b-a2a2-4270-b548-bf631d056fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:14.637288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-32886e5b-a2a2-4270-b548-bf631d056fb8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:14.638811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-32886e5b-a2a2-4270-b548-bf631d056fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:14.760943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1117/2249] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:14.779031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:14.781328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:14.854492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f28108-91c9-4785-9e23-75e2ccc0a102 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:14.859646 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdf09b-9551-42d5-b028-ddd1c2544273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:14.860316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdf09b-9551-42d5-b028-ddd1c2544273 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:14.860603 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdf09b-9551-42d5-b028-ddd1c2544273 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:14.860873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdf09b-9551-42d5-b028-ddd1c2544273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:14.867219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:14.876680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdf09b-9551-42d5-b028-ddd1c2544273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:14.878466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdf09b-9551-42d5-b028-ddd1c2544273 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:14.890163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1133/2250] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:48:15.223584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32886e5b-a2a2-4270-b548-bf631d056fb8 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:15.225093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-32886e5b-a2a2-4270-b548-bf631d056fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:15.250298 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32886e5b-a2a2-4270-b548-bf631d056fb8 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:15.256465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:15.257179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:15.257912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:15.258194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] PATH_INFO: `/v3/projects/2079f15807f04db1a080d438d12fc5fa` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:15.259573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2079f15807f04db1a080d438d12fc5fa)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:15.261401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:15.369597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2079f15807f04db1a080d438d12fc5fa', 'request_id': 'req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:15.376608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2079f15807f04db1a080d438d12fc5fa', 'request_id': 'req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:15.384524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2079f15807f04db1a080d438d12fc5fa', 'request_id': 'req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:15.418727 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] The token cache is being invalidate because project 2079f15807f04db1a080d438d12fc5fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:48:15.419026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:15.419718 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:15.421438 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1118/2251] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:48:15 2022] DELETE /identity/v3/projects/2079f15807f04db1a080d438d12fc5fa => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:15.425877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdf09b-9551-42d5-b028-ddd1c2544273 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:15.439917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdf09b-9551-42d5-b028-ddd1c2544273 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:15.551745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:15.552383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:15.552811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:15.553095 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] PATH_INFO: `/v3/users/cd1d6f7834e14e1daa1e4aa5f8f4525b` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:15.554370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd1d6f7834e14e1daa1e4aa5f8f4525b)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:15.556392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:15.617487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd1d6f7834e14e1daa1e4aa5f8f4525b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:15.622415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd1d6f7834e14e1daa1e4aa5f8f4525b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:15.633564 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd1d6f7834e14e1daa1e4aa5f8f4525b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:15.634596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd1d6f7834e14e1daa1e4aa5f8f4525b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:15.665085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd1d6f7834e14e1daa1e4aa5f8f4525b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:15.684536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1134/2252] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:48:15 2022] DELETE /identity/v3/users/cd1d6f7834e14e1daa1e4aa5f8f4525b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:16.019542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:16.020991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:16.039280 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745611a-2ac3-4041-bd8c-82f9eac1fe8a admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:16.103781 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:16.104723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:16.105011 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:16.105309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] PATH_INFO: `/v3/projects/877556067d064c8c974c29b7fc7e0c81` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:16.106665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=877556067d064c8c974c29b7fc7e0c81)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:16.108633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:16.116473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:16.168401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '877556067d064c8c974c29b7fc7e0c81', 'request_id': 'req-9f2ede37-71ef-408f-a1da-73212ceb71a2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:16.175491 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '877556067d064c8c974c29b7fc7e0c81', 'request_id': 'req-9f2ede37-71ef-408f-a1da-73212ceb71a2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:16.184385 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '877556067d064c8c974c29b7fc7e0c81', 'request_id': 'req-9f2ede37-71ef-408f-a1da-73212ceb71a2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:16.199489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] The token cache is being invalidate because project 877556067d064c8c974c29b7fc7e0c81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:48:16.199788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:16.200475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:16.204877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1119/2253] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:48:16 2022] DELETE /identity/v3/projects/877556067d064c8c974c29b7fc7e0c81 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:16.206685 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:16.216534 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:16.241182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b0aa7-90ca-4964-820a-4e16ab72c570 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:16.309062 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b99edb-3d69-46ea-a1b3-caf3b0afbf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:16.309556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b99edb-3d69-46ea-a1b3-caf3b0afbf5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:16.309996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b99edb-3d69-46ea-a1b3-caf3b0afbf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:16.310274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b99edb-3d69-46ea-a1b3-caf3b0afbf5f admin admin] PATH_INFO: `/v3/users/1208d0817edb4fa6ab8b14fd30c27bde` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:16.311544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b99edb-3d69-46ea-a1b3-caf3b0afbf5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1208d0817edb4fa6ab8b14fd30c27bde)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:16.313583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b99edb-3d69-46ea-a1b3-caf3b0afbf5f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:16.328795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:16.374760 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-18b99edb-3d69-46ea-a1b3-caf3b0afbf5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1208d0817edb4fa6ab8b14fd30c27bde'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:16.375502 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-18b99edb-3d69-46ea-a1b3-caf3b0afbf5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1208d0817edb4fa6ab8b14fd30c27bde'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:16.383828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-18b99edb-3d69-46ea-a1b3-caf3b0afbf5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1208d0817edb4fa6ab8b14fd30c27bde'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:16.384602 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-18b99edb-3d69-46ea-a1b3-caf3b0afbf5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1208d0817edb4fa6ab8b14fd30c27bde'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:16.400567 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-18b99edb-3d69-46ea-a1b3-caf3b0afbf5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1208d0817edb4fa6ab8b14fd30c27bde'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:16.418530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1135/2254] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:48:16 2022] DELETE /identity/v3/users/1208d0817edb4fa6ab8b14fd30c27bde => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:16.818458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:16.819976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:16.840418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ede37-71ef-408f-a1da-73212ceb71a2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:16.945114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c169ac0-628b-457a-9444-e6f535012656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:16.945114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c169ac0-628b-457a-9444-e6f535012656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:16.945114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c169ac0-628b-457a-9444-e6f535012656 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:16.945114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c169ac0-628b-457a-9444-e6f535012656 admin admin] PATH_INFO: `/v3/projects/12037595ad034bcab1a7e371c8cce028` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:16.946591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c169ac0-628b-457a-9444-e6f535012656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12037595ad034bcab1a7e371c8cce028)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:16.948604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c169ac0-628b-457a-9444-e6f535012656 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:17.010313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4c169ac0-628b-457a-9444-e6f535012656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12037595ad034bcab1a7e371c8cce028', 'request_id': 'req-4c169ac0-628b-457a-9444-e6f535012656'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:17.016919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4c169ac0-628b-457a-9444-e6f535012656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12037595ad034bcab1a7e371c8cce028', 'request_id': 'req-4c169ac0-628b-457a-9444-e6f535012656'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:17.026587 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4c169ac0-628b-457a-9444-e6f535012656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12037595ad034bcab1a7e371c8cce028', 'request_id': 'req-4c169ac0-628b-457a-9444-e6f535012656'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:17.040429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4c169ac0-628b-457a-9444-e6f535012656 admin admin] The token cache is being invalidate because project 12037595ad034bcab1a7e371c8cce028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:48:17.040722 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4c169ac0-628b-457a-9444-e6f535012656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:17.041345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4c169ac0-628b-457a-9444-e6f535012656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:17.043201 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1120/2255] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:48:16 2022] DELETE /identity/v3/projects/12037595ad034bcab1a7e371c8cce028 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:17.050439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ea98a-52ef-4a14-ad73-e4cbe533a439 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:17.050948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ea98a-52ef-4a14-ad73-e4cbe533a439 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:17.051118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ea98a-52ef-4a14-ad73-e4cbe533a439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:17.073079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-3a9ea98a-52ef-4a14-ad73-e4cbe533a439 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:17.074379 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3a9ea98a-52ef-4a14-ad73-e4cbe533a439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:17.147992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1136/2256] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:17.156534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bf33d-d86d-4310-b1ea-d88a198f2be4 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:17.156827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bf33d-d86d-4310-b1ea-d88a198f2be4 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:17.157108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bf33d-d86d-4310-b1ea-d88a198f2be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:17.181343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-ac2bf33d-d86d-4310-b1ea-d88a198f2be4 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:17.182604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ac2bf33d-d86d-4310-b1ea-d88a198f2be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:17.192744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1121/2257] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:17.197525 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ea98a-52ef-4a14-ad73-e4cbe533a439 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:17.207894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ea98a-52ef-4a14-ad73-e4cbe533a439 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:17.212801 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e2a1a-4202-494c-a147-9ad7cb5f424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:17.213460 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e2a1a-4202-494c-a147-9ad7cb5f424c admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:17.213777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e2a1a-4202-494c-a147-9ad7cb5f424c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:17.214062 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e2a1a-4202-494c-a147-9ad7cb5f424c admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:17.214773 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e2a1a-4202-494c-a147-9ad7cb5f424c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:48:17.215084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e2a1a-4202-494c-a147-9ad7cb5f424c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:17.216733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e2a1a-4202-494c-a147-9ad7cb5f424c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:17.233814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1137/2258] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:48:17.234957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2bf33d-d86d-4310-b1ea-d88a198f2be4 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:17.247173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2bf33d-d86d-4310-b1ea-d88a198f2be4 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:17.252627 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc95723-35c5-4aa6-822c-1fc228a1f709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:17.253272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc95723-35c5-4aa6-822c-1fc228a1f709 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:17.253559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc95723-35c5-4aa6-822c-1fc228a1f709 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:17.253878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc95723-35c5-4aa6-822c-1fc228a1f709 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:17.254516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc95723-35c5-4aa6-822c-1fc228a1f709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:17.256441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc95723-35c5-4aa6-822c-1fc228a1f709 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:17.274604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:17.291771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1122/2259] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:48:17 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:48:17.295807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e2a1a-4202-494c-a147-9ad7cb5f424c admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:17.308889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e2a1a-4202-494c-a147-9ad7cb5f424c admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:17.378246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb935a9-86ed-40c7-b5f2-acd1d4d9c206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:17.380478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb935a9-86ed-40c7-b5f2-acd1d4d9c206 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:17.380478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb935a9-86ed-40c7-b5f2-acd1d4d9c206 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:17.380478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb935a9-86ed-40c7-b5f2-acd1d4d9c206 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:17.380478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb935a9-86ed-40c7-b5f2-acd1d4d9c206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:17.382471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb935a9-86ed-40c7-b5f2-acd1d4d9c206 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:17.410909 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1138/2260] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:48:17 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:48:17.411993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc95723-35c5-4aa6-822c-1fc228a1f709 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:17.425931 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc95723-35c5-4aa6-822c-1fc228a1f709 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:17.431081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33e445-e4c8-46b2-952c-1305bd3f9106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:17.431703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33e445-e4c8-46b2-952c-1305bd3f9106 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:17.431983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33e445-e4c8-46b2-952c-1305bd3f9106 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:17.432378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33e445-e4c8-46b2-952c-1305bd3f9106 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:17.433041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33e445-e4c8-46b2-952c-1305bd3f9106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:17.434681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33e445-e4c8-46b2-952c-1305bd3f9106 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:17.449898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1123/2261] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:48:17.451134 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb935a9-86ed-40c7-b5f2-acd1d4d9c206 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:17.463635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb935a9-86ed-40c7-b5f2-acd1d4d9c206 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:17.468688 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f1aae-439e-4ff4-8073-f7607fedfd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:17.469332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f1aae-439e-4ff4-8073-f7607fedfd97 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:17.469613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f1aae-439e-4ff4-8073-f7607fedfd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:17.469940 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f1aae-439e-4ff4-8073-f7607fedfd97 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:17.470666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f1aae-439e-4ff4-8073-f7607fedfd97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:17.472276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f1aae-439e-4ff4-8073-f7607fedfd97 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:17.474771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:17.491445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1139/2262] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:48:17.492009 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33e445-e4c8-46b2-952c-1305bd3f9106 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:17.504261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33e445-e4c8-46b2-952c-1305bd3f9106 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:17.509342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee73ac-befe-4ce8-8ab3-e2017ce004a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:17.510036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee73ac-befe-4ce8-8ab3-e2017ce004a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:17.510316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee73ac-befe-4ce8-8ab3-e2017ce004a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:17.510590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee73ac-befe-4ce8-8ab3-e2017ce004a6 admin admin] PATH_INFO: `/v3/projects/7ea5bd24cf1b44b38e0254e543539033/users/a158977d001c41eba0e47b40ffbb2f89/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:17.529352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee73ac-befe-4ce8-8ab3-e2017ce004a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea5bd24cf1b44b38e0254e543539033, user_id=a158977d001c41eba0e47b40ffbb2f89, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:17.534173 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee73ac-befe-4ce8-8ab3-e2017ce004a6 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:17.564311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1124/2263] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:48:17 2022] PUT /identity/v3/projects/7ea5bd24cf1b44b38e0254e543539033/users/a158977d001c41eba0e47b40ffbb2f89/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:48:17.566701 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff0b59-7d1a-47c7-a5a0-595a4f2b3b91 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:17.566993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff0b59-7d1a-47c7-a5a0-595a4f2b3b91 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:17.567272 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff0b59-7d1a-47c7-a5a0-595a4f2b3b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:17.599328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d9ff0b59-7d1a-47c7-a5a0-595a4f2b3b91 None None] MFA Rules not processed for user `a158977d001c41eba0e47b40ffbb2f89`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:17.600601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d9ff0b59-7d1a-47c7-a5a0-595a4f2b3b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:17.695195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1140/2264] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:17.706865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee73ac-befe-4ce8-8ab3-e2017ce004a6 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:17.718350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-73ee73ac-befe-4ce8-8ab3-e2017ce004a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:17.828472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee73ac-befe-4ce8-8ab3-e2017ce004a6 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:17.832910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f6972-58e2-43d3-8f38-193ee36441a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:17.833638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f6972-58e2-43d3-8f38-193ee36441a3 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:17.834338 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f6972-58e2-43d3-8f38-193ee36441a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:17.834638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f6972-58e2-43d3-8f38-193ee36441a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:17.858541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f6972-58e2-43d3-8f38-193ee36441a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:17.860793 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f6972-58e2-43d3-8f38-193ee36441a3 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:17.873966 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1125/2265] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:48:19.473152 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c54063-83c3-4fd3-92ae-651f3d25e5c2 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:19.473767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c54063-83c3-4fd3-92ae-651f3d25e5c2 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:19.474020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c54063-83c3-4fd3-92ae-651f3d25e5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:19.494468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-a1c54063-83c3-4fd3-92ae-651f3d25e5c2 None None] MFA Rules not processed for user `8fcf4142aa8340d692b1b3ed0aee7c87`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:19.496038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a1c54063-83c3-4fd3-92ae-651f3d25e5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:19.580960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1141/2266] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:20.632730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f6972-58e2-43d3-8f38-193ee36441a3 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:20.720557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f6972-58e2-43d3-8f38-193ee36441a3 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:20.725596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:20.726295 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:20.726568 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:20.726850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] PATH_INFO: `/v3/users/a158977d001c41eba0e47b40ffbb2f89` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:20.729750 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a158977d001c41eba0e47b40ffbb2f89)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:20.732194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:20.791151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a158977d001c41eba0e47b40ffbb2f89'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:20.791820 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a158977d001c41eba0e47b40ffbb2f89'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:20.819691 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a158977d001c41eba0e47b40ffbb2f89'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:20.821817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:20.832048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a158977d001c41eba0e47b40ffbb2f89'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:20.833488 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a158977d001c41eba0e47b40ffbb2f89'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:20.853053 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1126/2267] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:48:20 2022] DELETE /identity/v3/users/a158977d001c41eba0e47b40ffbb2f89 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:48:20.857675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4ccca-e966-4c2c-9f70-64a1c3355794 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:20.858175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4ccca-e966-4c2c-9f70-64a1c3355794 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:20.858292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4ccca-e966-4c2c-9f70-64a1c3355794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:20.877621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-91a4ccca-e966-4c2c-9f70-64a1c3355794 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:20.878901 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-91a4ccca-e966-4c2c-9f70-64a1c3355794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:20.942706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1142/2268] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:20.953339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:20.955472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:21.020178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138de3d-e3f7-49fb-b372-9b8f995e9dd2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:21.024257 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11205204-cef2-4637-bac5-7d2f9f7f2fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:21.025065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11205204-cef2-4637-bac5-7d2f9f7f2fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:21.025494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11205204-cef2-4637-bac5-7d2f9f7f2fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:21.025990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11205204-cef2-4637-bac5-7d2f9f7f2fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:21.036232 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11205204-cef2-4637-bac5-7d2f9f7f2fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:21.037928 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11205204-cef2-4637-bac5-7d2f9f7f2fd6 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:21.048572 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1127/2269] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:48:21.846915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4ccca-e966-4c2c-9f70-64a1c3355794 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:21.849123 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-91a4ccca-e966-4c2c-9f70-64a1c3355794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:21.873580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4ccca-e966-4c2c-9f70-64a1c3355794 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:21.879396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:21.880045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:21.880326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:21.880655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56 admin admin] PATH_INFO: `/v3/projects/7ea5bd24cf1b44b38e0254e543539033` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:21.881918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea5bd24cf1b44b38e0254e543539033)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:21.883895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:21.924276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:21.940187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea5bd24cf1b44b38e0254e543539033', 'request_id': 'req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:21.946377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea5bd24cf1b44b38e0254e543539033', 'request_id': 'req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:21.951981 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea5bd24cf1b44b38e0254e543539033', 'request_id': 'req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:21.964617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56 admin admin] The token cache is being invalidate because project 7ea5bd24cf1b44b38e0254e543539033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:48:21.964940 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:21.965571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5088bbb2-6c8f-47db-85e5-f6cbdbe35e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:21.967089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1143/2270] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:48:21 2022] DELETE /identity/v3/projects/7ea5bd24cf1b44b38e0254e543539033 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:21.973934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708d199-1b14-4e13-8beb-3f6e52a0e9c0 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:21.974436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708d199-1b14-4e13-8beb-3f6e52a0e9c0 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:21.974546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708d199-1b14-4e13-8beb-3f6e52a0e9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:21.993577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-2708d199-1b14-4e13-8beb-3f6e52a0e9c0 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:21.994811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2708d199-1b14-4e13-8beb-3f6e52a0e9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:22.052815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1128/2271] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:22.060469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ec8f5-4ae6-4bd2-8b34-3f70886e05de None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:22.060758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ec8f5-4ae6-4bd2-8b34-3f70886e05de None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:22.061033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ec8f5-4ae6-4bd2-8b34-3f70886e05de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:22.081223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-9c6ec8f5-4ae6-4bd2-8b34-3f70886e05de None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:22.082527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9c6ec8f5-4ae6-4bd2-8b34-3f70886e05de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:22.090201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1144/2272] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:22.094601 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708d199-1b14-4e13-8beb-3f6e52a0e9c0 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:22.104632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708d199-1b14-4e13-8beb-3f6e52a0e9c0 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:22.109286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03973f55-4c84-4704-a29c-5b89ddd734b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:22.109917 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03973f55-4c84-4704-a29c-5b89ddd734b2 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:22.110201 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03973f55-4c84-4704-a29c-5b89ddd734b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:22.110479 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03973f55-4c84-4704-a29c-5b89ddd734b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:22.111147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03973f55-4c84-4704-a29c-5b89ddd734b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:48:22.111454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03973f55-4c84-4704-a29c-5b89ddd734b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:22.113080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03973f55-4c84-4704-a29c-5b89ddd734b2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:22.124470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1129/2273] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:48:22.129160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ec8f5-4ae6-4bd2-8b34-3f70886e05de None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:22.138307 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ec8f5-4ae6-4bd2-8b34-3f70886e05de None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:22.143653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b269664-66a6-40ed-9b03-731d3ec34a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:22.144241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b269664-66a6-40ed-9b03-731d3ec34a9f admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:22.144518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b269664-66a6-40ed-9b03-731d3ec34a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:22.144786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b269664-66a6-40ed-9b03-731d3ec34a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:22.145414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b269664-66a6-40ed-9b03-731d3ec34a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:22.147469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b269664-66a6-40ed-9b03-731d3ec34a9f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:22.173289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1145/2274] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:48:22 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:48:22.177284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03973f55-4c84-4704-a29c-5b89ddd734b2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:22.187335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03973f55-4c84-4704-a29c-5b89ddd734b2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:22.243868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5a80f-cddd-46f4-bbe0-554bfebe5153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:22.244475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5a80f-cddd-46f4-bbe0-554bfebe5153 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:22.244754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5a80f-cddd-46f4-bbe0-554bfebe5153 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:22.245021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5a80f-cddd-46f4-bbe0-554bfebe5153 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:22.245658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5a80f-cddd-46f4-bbe0-554bfebe5153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:22.247761 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5a80f-cddd-46f4-bbe0-554bfebe5153 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:22.271552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1130/2275] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:48:22 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:48:22.275671 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b269664-66a6-40ed-9b03-731d3ec34a9f admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:22.289546 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b269664-66a6-40ed-9b03-731d3ec34a9f admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:22.294297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e04aa-fafc-43b6-9e27-a61c9be95ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:22.294896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e04aa-fafc-43b6-9e27-a61c9be95ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:22.295176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e04aa-fafc-43b6-9e27-a61c9be95ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:22.295450 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e04aa-fafc-43b6-9e27-a61c9be95ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:22.296101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e04aa-fafc-43b6-9e27-a61c9be95ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:22.298413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e04aa-fafc-43b6-9e27-a61c9be95ff2 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:22.308589 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1146/2276] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:48:22.312677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5a80f-cddd-46f4-bbe0-554bfebe5153 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:22.318281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:22.324667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5a80f-cddd-46f4-bbe0-554bfebe5153 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:22.330647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf0790-dab2-4bdb-861f-35e417db12ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:22.331252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf0790-dab2-4bdb-861f-35e417db12ab admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:22.331538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf0790-dab2-4bdb-861f-35e417db12ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:22.331813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf0790-dab2-4bdb-861f-35e417db12ab admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:22.332549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf0790-dab2-4bdb-861f-35e417db12ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:22.334211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf0790-dab2-4bdb-861f-35e417db12ab admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:22.345047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1131/2277] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:48:22.349764 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e04aa-fafc-43b6-9e27-a61c9be95ff2 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:22.359503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e04aa-fafc-43b6-9e27-a61c9be95ff2 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:22.364698 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d9f83-0499-412d-8585-a2ea5986d346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:22.365335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d9f83-0499-412d-8585-a2ea5986d346 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:22.365614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d9f83-0499-412d-8585-a2ea5986d346 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:22.365921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d9f83-0499-412d-8585-a2ea5986d346 admin admin] PATH_INFO: `/v3/projects/6743f1434e0c44c58b66c23758c6b26c/users/8dd19844068e4ce9a35235db8deb0044/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:22.380022 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d9f83-0499-412d-8585-a2ea5986d346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6743f1434e0c44c58b66c23758c6b26c, user_id=8dd19844068e4ce9a35235db8deb0044, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:22.382704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d9f83-0499-412d-8585-a2ea5986d346 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:22.395851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1147/2278] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:48:22 2022] PUT /identity/v3/projects/6743f1434e0c44c58b66c23758c6b26c/users/8dd19844068e4ce9a35235db8deb0044/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:22.406367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f2039-8c18-4aec-9ed8-f3677b2dd6e2 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:22.406652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f2039-8c18-4aec-9ed8-f3677b2dd6e2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:22.406924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f2039-8c18-4aec-9ed8-f3677b2dd6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:22.434310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-7e6f2039-8c18-4aec-9ed8-f3677b2dd6e2 None None] MFA Rules not processed for user `8dd19844068e4ce9a35235db8deb0044`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:22.435497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7e6f2039-8c18-4aec-9ed8-f3677b2dd6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:22.496684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1132/2279] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:22.507776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d9f83-0499-412d-8585-a2ea5986d346 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:22.509520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e29d9f83-0499-412d-8585-a2ea5986d346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:22.565418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d9f83-0499-412d-8585-a2ea5986d346 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:22.568960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947d377-5e58-4a02-95c7-8e5acb51fcc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:22.569551 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947d377-5e58-4a02-95c7-8e5acb51fcc5 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:22.569857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947d377-5e58-4a02-95c7-8e5acb51fcc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:22.570131 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947d377-5e58-4a02-95c7-8e5acb51fcc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:22.578085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947d377-5e58-4a02-95c7-8e5acb51fcc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:22.579529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947d377-5e58-4a02-95c7-8e5acb51fcc5 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:22.588315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1148/2280] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:48:27.149581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dca76-db82-4403-8dc9-76515712fe98 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:27.150129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dca76-db82-4403-8dc9-76515712fe98 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:27.150736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0dca76-db82-4403-8dc9-76515712fe98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:27.171859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-6e0dca76-db82-4403-8dc9-76515712fe98 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:27.173092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6e0dca76-db82-4403-8dc9-76515712fe98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:27.236550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1133/2281] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:48:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:27.248356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947d377-5e58-4a02-95c7-8e5acb51fcc5 service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:27.250877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-5947d377-5e58-4a02-95c7-8e5acb51fcc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:27.355911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:27.381245 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947d377-5e58-4a02-95c7-8e5acb51fcc5 service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:27.386378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a3847-6459-49bf-bbd6-ba02f79e3967 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:27.387033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a3847-6459-49bf-bbd6-ba02f79e3967 service swift] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:27.387317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a3847-6459-49bf-bbd6-ba02f79e3967 service swift] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:27.387592 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a3847-6459-49bf-bbd6-ba02f79e3967 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:27.398427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a3847-6459-49bf-bbd6-ba02f79e3967 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:27.400654 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a3847-6459-49bf-bbd6-ba02f79e3967 service swift] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:27.411000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1149/2282] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:48:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:48:37.228308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=61 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:37.406377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=7 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:48:59.409068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dca76-db82-4403-8dc9-76515712fe98 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:59.410553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6e0dca76-db82-4403-8dc9-76515712fe98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:59.474936 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0dca76-db82-4403-8dc9-76515712fe98 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:59.479425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:59.480048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:59.480326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:59.480599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] PATH_INFO: `/v3/users/18bce21cd99842e8abef570dce4fcdb3` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:59.481755 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18bce21cd99842e8abef570dce4fcdb3)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:59.483677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:59.530023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18bce21cd99842e8abef570dce4fcdb3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:59.530787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18bce21cd99842e8abef570dce4fcdb3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:59.542859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18bce21cd99842e8abef570dce4fcdb3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:59.550037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18bce21cd99842e8abef570dce4fcdb3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:59.550768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18bce21cd99842e8abef570dce4fcdb3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:48:59.564000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1134/2283] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:48:59 2022] DELETE /identity/v3/users/18bce21cd99842e8abef570dce4fcdb3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:48:59.568542 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fea16-f370-467c-b3cc-83f33d2bead9 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:59.569071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fea16-f370-467c-b3cc-83f33d2bead9 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:59.569179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fea16-f370-467c-b3cc-83f33d2bead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:59.589714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-fb7fea16-f370-467c-b3cc-83f33d2bead9 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:48:59.590955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fb7fea16-f370-467c-b3cc-83f33d2bead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:59.656990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1150/2284] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:48:59.670531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:48:59.672593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:48:59.745826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4761c66-67b4-42f1-97ba-4eea16b903e8 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:48:59.750370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22dd969-4847-4da0-a1f2-e1c751fa1387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:48:59.750983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22dd969-4847-4da0-a1f2-e1c751fa1387 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:48:59.751263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22dd969-4847-4da0-a1f2-e1c751fa1387 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:48:59.751538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22dd969-4847-4da0-a1f2-e1c751fa1387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:48:59.765964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22dd969-4847-4da0-a1f2-e1c751fa1387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:48:59.765964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22dd969-4847-4da0-a1f2-e1c751fa1387 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:48:59.777265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1135/2285] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:48:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:00.064140 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fea16-f370-467c-b3cc-83f33d2bead9 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:00.065946 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fb7fea16-f370-467c-b3cc-83f33d2bead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:00.080824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fea16-f370-467c-b3cc-83f33d2bead9 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:00.095228 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:00.096172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:00.096626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:00.096967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0 admin admin] PATH_INFO: `/v3/projects/bf80faf02bc24c388aa84f92f64681e5` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:00.098666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf80faf02bc24c388aa84f92f64681e5)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:00.101023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:00.157006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf80faf02bc24c388aa84f92f64681e5', 'request_id': 'req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:00.163207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf80faf02bc24c388aa84f92f64681e5', 'request_id': 'req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:00.168668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf80faf02bc24c388aa84f92f64681e5', 'request_id': 'req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:00.181332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0 admin admin] The token cache is being invalidate because project bf80faf02bc24c388aa84f92f64681e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:49:00.181718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:00.182504 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-fe9a21f1-3037-41f9-b80e-e10a76ee61c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:00.184579 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1151/2286] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:49:00 2022] DELETE /identity/v3/projects/bf80faf02bc24c388aa84f92f64681e5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:49:00.196669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764e37-ad47-476d-a416-6a6ea7b2d156 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:00.196669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764e37-ad47-476d-a416-6a6ea7b2d156 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:00.196669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764e37-ad47-476d-a416-6a6ea7b2d156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:00.210960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-98764e37-ad47-476d-a416-6a6ea7b2d156 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:00.212160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-98764e37-ad47-476d-a416-6a6ea7b2d156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:00.229619 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:00.275544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1136/2287] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:00.283831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6263a-aa97-4ab5-a09f-82f9576420d2 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:00.284195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6263a-aa97-4ab5-a09f-82f9576420d2 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:00.284534 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6263a-aa97-4ab5-a09f-82f9576420d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:00.305189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-0da6263a-aa97-4ab5-a09f-82f9576420d2 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:00.306581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-0da6263a-aa97-4ab5-a09f-82f9576420d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:00.315283 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1152/2288] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:00.319515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764e37-ad47-476d-a416-6a6ea7b2d156 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:00.329678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764e37-ad47-476d-a416-6a6ea7b2d156 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:00.337372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ead47-b144-47e4-9daa-2f2b6788b92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:00.338327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ead47-b144-47e4-9daa-2f2b6788b92a admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:00.338634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ead47-b144-47e4-9daa-2f2b6788b92a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:00.338915 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ead47-b144-47e4-9daa-2f2b6788b92a admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:00.339574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ead47-b144-47e4-9daa-2f2b6788b92a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:49:00.339880 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ead47-b144-47e4-9daa-2f2b6788b92a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:00.341751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ead47-b144-47e4-9daa-2f2b6788b92a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:00.354450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1137/2289] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:49:00.358867 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6263a-aa97-4ab5-a09f-82f9576420d2 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:00.368520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6263a-aa97-4ab5-a09f-82f9576420d2 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:00.373477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cb23c-b8c2-4e83-9f1c-e19ab65a2104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:00.374162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cb23c-b8c2-4e83-9f1c-e19ab65a2104 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:00.374998 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cb23c-b8c2-4e83-9f1c-e19ab65a2104 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:00.375298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cb23c-b8c2-4e83-9f1c-e19ab65a2104 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:00.375980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cb23c-b8c2-4e83-9f1c-e19ab65a2104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:00.378280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cb23c-b8c2-4e83-9f1c-e19ab65a2104 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:00.413302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1153/2290] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:49:00 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:49:00.417572 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ead47-b144-47e4-9daa-2f2b6788b92a admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:00.430965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ead47-b144-47e4-9daa-2f2b6788b92a admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:00.489431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074dbb6-2637-40fe-bfb6-947c7b8ca5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:00.490070 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074dbb6-2637-40fe-bfb6-947c7b8ca5a5 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:00.490350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074dbb6-2637-40fe-bfb6-947c7b8ca5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:00.490622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074dbb6-2637-40fe-bfb6-947c7b8ca5a5 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:00.491247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074dbb6-2637-40fe-bfb6-947c7b8ca5a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:00.493189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074dbb6-2637-40fe-bfb6-947c7b8ca5a5 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:00.515276 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1138/2291] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:49:00 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:49:00.519551 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cb23c-b8c2-4e83-9f1c-e19ab65a2104 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:00.529711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cb23c-b8c2-4e83-9f1c-e19ab65a2104 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:00.534911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80e9ca-5e7b-48cb-a9a0-5da083780b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:00.535496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80e9ca-5e7b-48cb-a9a0-5da083780b0a admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:00.535773 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80e9ca-5e7b-48cb-a9a0-5da083780b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:00.536043 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80e9ca-5e7b-48cb-a9a0-5da083780b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:00.536692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80e9ca-5e7b-48cb-a9a0-5da083780b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:00.538304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80e9ca-5e7b-48cb-a9a0-5da083780b0a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:00.548081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1154/2292] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:49:00.551942 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074dbb6-2637-40fe-bfb6-947c7b8ca5a5 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:00.564660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074dbb6-2637-40fe-bfb6-947c7b8ca5a5 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:00.569394 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1a13c-1e80-4a38-8f64-d1f211ff0cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:00.570038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1a13c-1e80-4a38-8f64-d1f211ff0cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:00.570320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1a13c-1e80-4a38-8f64-d1f211ff0cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:00.570592 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1a13c-1e80-4a38-8f64-d1f211ff0cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:00.571239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1a13c-1e80-4a38-8f64-d1f211ff0cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:00.572881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1a13c-1e80-4a38-8f64-d1f211ff0cc7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:00.584740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1139/2293] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:49:00.588896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80e9ca-5e7b-48cb-a9a0-5da083780b0a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:00.599689 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80e9ca-5e7b-48cb-a9a0-5da083780b0a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:00.604453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bdff7-8ce3-458b-b92d-238c051dd7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:00.605107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bdff7-8ce3-458b-b92d-238c051dd7db admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:00.605385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bdff7-8ce3-458b-b92d-238c051dd7db admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:00.605658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936bdff7-8ce3-458b-b92d-238c051dd7db admin admin] PATH_INFO: `/v3/projects/1104ee1508e24a2b87820a8b0fafd0e1/users/f928852d90b94ac5979e4a7806896b7a/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:00.620985 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936bdff7-8ce3-458b-b92d-238c051dd7db admin admin] RBAC: Authorizing `identity:create_grant(project_id=1104ee1508e24a2b87820a8b0fafd0e1, user_id=f928852d90b94ac5979e4a7806896b7a, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:00.623678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936bdff7-8ce3-458b-b92d-238c051dd7db admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:00.637425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1155/2294] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:49:00 2022] PUT /identity/v3/projects/1104ee1508e24a2b87820a8b0fafd0e1/users/f928852d90b94ac5979e4a7806896b7a/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:49:00.642495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee6a7f-8038-4057-a9f9-e3d861a5464a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:00.642787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee6a7f-8038-4057-a9f9-e3d861a5464a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:00.643065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee6a7f-8038-4057-a9f9-e3d861a5464a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:00.663221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-31ee6a7f-8038-4057-a9f9-e3d861a5464a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:00.664500 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-31ee6a7f-8038-4057-a9f9-e3d861a5464a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:00.726580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1140/2295] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:00.736696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bdff7-8ce3-458b-b92d-238c051dd7db admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:00.738412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-936bdff7-8ce3-458b-b92d-238c051dd7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:00.760414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:00.803769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936bdff7-8ce3-458b-b92d-238c051dd7db admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:00.807826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c5daa-4403-4eb1-bdd9-8ff1eb442b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:00.808437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c5daa-4403-4eb1-bdd9-8ff1eb442b0b service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:00.808719 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c5daa-4403-4eb1-bdd9-8ff1eb442b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:00.808994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c5daa-4403-4eb1-bdd9-8ff1eb442b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:00.824390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c5daa-4403-4eb1-bdd9-8ff1eb442b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:00.824390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c5daa-4403-4eb1-bdd9-8ff1eb442b0b service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:00.830233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1156/2296] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:01.897557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e644d-1344-42f0-aa27-d5035512467e None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:01.898652 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e644d-1344-42f0-aa27-d5035512467e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:01.899021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e644d-1344-42f0-aa27-d5035512467e None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:01.900394 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1141/2297] 10.222.0.26 () {54 vars in 961 bytes} [Tue Oct 4 08:49:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:49:01.906348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441563f-6b66-44e6-8317-1b4de8b963d8 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:01.907065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441563f-6b66-44e6-8317-1b4de8b963d8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:01.907065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441563f-6b66-44e6-8317-1b4de8b963d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:01.929300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-9441563f-6b66-44e6-8317-1b4de8b963d8 None None] MFA Rules not processed for user `abd158648914409f87ce0599843f0fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:01.930604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9441563f-6b66-44e6-8317-1b4de8b963d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:02.010451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1157/2298] 10.222.0.26 () {58 vars in 1058 bytes} [Tue Oct 4 08:49:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:02.021121 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e644d-1344-42f0-aa27-d5035512467e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:02.039544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e644d-1344-42f0-aa27-d5035512467e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:02.044553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b7e1c-1fc6-404c-8fae-5892ff6fd4ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:02.045151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b7e1c-1fc6-404c-8fae-5892ff6fd4ed service placement] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:02.045427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b7e1c-1fc6-404c-8fae-5892ff6fd4ed service placement] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:02.045715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b7e1c-1fc6-404c-8fae-5892ff6fd4ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:02.047894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4e1b7e1c-1fc6-404c-8fae-5892ff6fd4ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:02.065041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b7e1c-1fc6-404c-8fae-5892ff6fd4ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:02.067231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b7e1c-1fc6-404c-8fae-5892ff6fd4ed service placement] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:02.093424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1142/2299] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:04.637609 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682430c1-ab4c-4e93-a7bc-36c1cbfd3723 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:04.638750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682430c1-ab4c-4e93-a7bc-36c1cbfd3723 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:04.639247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682430c1-ab4c-4e93-a7bc-36c1cbfd3723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:04.665839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-682430c1-ab4c-4e93-a7bc-36c1cbfd3723 None None] MFA Rules not processed for user `efea00dacdca4d1d8e919fb446e164ea`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:04.667508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-682430c1-ab4c-4e93-a7bc-36c1cbfd3723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:04.776899 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1158/2300] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:49:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:05.948643 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3b9c7-879a-4f71-8ab7-ce029d6a3960 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:05.949143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3b9c7-879a-4f71-8ab7-ce029d6a3960 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:05.949241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3b9c7-879a-4f71-8ab7-ce029d6a3960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:05.992169 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-ebd3b9c7-879a-4f71-8ab7-ce029d6a3960 None None] MFA Rules not processed for user `f928852d90b94ac5979e4a7806896b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:05.993724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ebd3b9c7-879a-4f71-8ab7-ce029d6a3960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:06.000227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:06.081945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1143/2301] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:49:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:06.086969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682430c1-ab4c-4e93-a7bc-36c1cbfd3723 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:06.103728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682430c1-ab4c-4e93-a7bc-36c1cbfd3723 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:06.109947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261caff6-ef5b-4e33-b880-83a932e3aee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:06.110562 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261caff6-ef5b-4e33-b880-83a932e3aee7 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:06.110840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261caff6-ef5b-4e33-b880-83a932e3aee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:06.111113 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261caff6-ef5b-4e33-b880-83a932e3aee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:06.111751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261caff6-ef5b-4e33-b880-83a932e3aee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:06.113754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261caff6-ef5b-4e33-b880-83a932e3aee7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:06.157623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1159/2302] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:49:06 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:49:06.162305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3b9c7-879a-4f71-8ab7-ce029d6a3960 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:06.175287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3b9c7-879a-4f71-8ab7-ce029d6a3960 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:06.243932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bebc7c-9bbc-47e5-abed-c90f662afe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:06.244694 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bebc7c-9bbc-47e5-abed-c90f662afe07 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:06.245571 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bebc7c-9bbc-47e5-abed-c90f662afe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:06.245571 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bebc7c-9bbc-47e5-abed-c90f662afe07 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:06.246276 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bebc7c-9bbc-47e5-abed-c90f662afe07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:06.248220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bebc7c-9bbc-47e5-abed-c90f662afe07 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:06.277744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1144/2303] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:49:06 2022] POST /identity/v3/users => generated 296 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:49:06.279370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261caff6-ef5b-4e33-b880-83a932e3aee7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:06.297809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261caff6-ef5b-4e33-b880-83a932e3aee7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:06.304235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231cb0b-b37d-4059-ad11-8d74c910d891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:06.304816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231cb0b-b37d-4059-ad11-8d74c910d891 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:06.305139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231cb0b-b37d-4059-ad11-8d74c910d891 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:06.305394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231cb0b-b37d-4059-ad11-8d74c910d891 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:06.306155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231cb0b-b37d-4059-ad11-8d74c910d891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:06.307773 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231cb0b-b37d-4059-ad11-8d74c910d891 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:06.323984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1160/2304] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:49:06.330997 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bebc7c-9bbc-47e5-abed-c90f662afe07 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:06.350867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bebc7c-9bbc-47e5-abed-c90f662afe07 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:06.357163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f3ac9-5b8d-44b9-8a78-3580cd84fe19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:06.358379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f3ac9-5b8d-44b9-8a78-3580cd84fe19 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:06.358851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f3ac9-5b8d-44b9-8a78-3580cd84fe19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:06.359303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f3ac9-5b8d-44b9-8a78-3580cd84fe19 admin admin] PATH_INFO: `/v3/projects/e48a362e41a844ab80e24925dbf53993/users/6eb4145e013f49e9a37beeac46009a1b/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:06.377551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f3ac9-5b8d-44b9-8a78-3580cd84fe19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e48a362e41a844ab80e24925dbf53993, user_id=6eb4145e013f49e9a37beeac46009a1b, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:06.380340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f3ac9-5b8d-44b9-8a78-3580cd84fe19 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:06.395400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1145/2305] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:49:06 2022] PUT /identity/v3/projects/e48a362e41a844ab80e24925dbf53993/users/6eb4145e013f49e9a37beeac46009a1b/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:49:06.399529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231cb0b-b37d-4059-ad11-8d74c910d891 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:06.411931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231cb0b-b37d-4059-ad11-8d74c910d891 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:06.452317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:06.513893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec73aaa-4b37-44ce-a62f-7e717e8b69d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:06.514506 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec73aaa-4b37-44ce-a62f-7e717e8b69d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:06.514782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec73aaa-4b37-44ce-a62f-7e717e8b69d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:06.515053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec73aaa-4b37-44ce-a62f-7e717e8b69d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:06.515695 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec73aaa-4b37-44ce-a62f-7e717e8b69d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:06.517371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec73aaa-4b37-44ce-a62f-7e717e8b69d4 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:06.527473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1161/2306] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:49:06.532179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f3ac9-5b8d-44b9-8a78-3580cd84fe19 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:06.545952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f3ac9-5b8d-44b9-8a78-3580cd84fe19 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:06.551804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7db42a-0766-48df-8fc8-d2eae923a67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:06.552594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7db42a-0766-48df-8fc8-d2eae923a67c admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:06.553359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7db42a-0766-48df-8fc8-d2eae923a67c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:06.553937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7db42a-0766-48df-8fc8-d2eae923a67c admin admin] PATH_INFO: `/v3/projects/e48a362e41a844ab80e24925dbf53993/users/6eb4145e013f49e9a37beeac46009a1b/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:06.559115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7db42a-0766-48df-8fc8-d2eae923a67c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e48a362e41a844ab80e24925dbf53993, user_id=6eb4145e013f49e9a37beeac46009a1b, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:06.562485 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7db42a-0766-48df-8fc8-d2eae923a67c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:06.575231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1146/2307] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:49:06 2022] PUT /identity/v3/projects/e48a362e41a844ab80e24925dbf53993/users/6eb4145e013f49e9a37beeac46009a1b/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:49:12.819400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df032daa-6512-45b4-8346-bdb8aa406d58 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:12.819990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df032daa-6512-45b4-8346-bdb8aa406d58 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:12.819990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df032daa-6512-45b4-8346-bdb8aa406d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:12.855944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-df032daa-6512-45b4-8346-bdb8aa406d58 None None] MFA Rules not processed for user `6eb4145e013f49e9a37beeac46009a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:12.857207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-df032daa-6512-45b4-8346-bdb8aa406d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:12.940995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1162/2308] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:49:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:49:12.954814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7db42a-0766-48df-8fc8-d2eae923a67c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:12.956562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7e7db42a-0766-48df-8fc8-d2eae923a67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:13.017110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:13.038015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7db42a-0766-48df-8fc8-d2eae923a67c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:13.043715 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefafc1-ce6c-4f88-a765-3c4123bb3d4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:13.044340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefafc1-ce6c-4f88-a765-3c4123bb3d4d service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:13.044621 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefafc1-ce6c-4f88-a765-3c4123bb3d4d service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:13.044892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefafc1-ce6c-4f88-a765-3c4123bb3d4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:13.106742 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefafc1-ce6c-4f88-a765-3c4123bb3d4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:13.108921 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefafc1-ce6c-4f88-a765-3c4123bb3d4d service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:13.118781 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1147/2309] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:22.938086 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=49 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:23.114075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=30 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:24.759201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df032daa-6512-45b4-8346-bdb8aa406d58 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:24.759201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-df032daa-6512-45b4-8346-bdb8aa406d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:24.824668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df032daa-6512-45b4-8346-bdb8aa406d58 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:24.828480 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af70ed1-1125-4b07-a4b0-557b5e890f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:24.829111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af70ed1-1125-4b07-a4b0-557b5e890f97 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:24.829416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af70ed1-1125-4b07-a4b0-557b5e890f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:24.829655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af70ed1-1125-4b07-a4b0-557b5e890f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:24.831908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-5af70ed1-1125-4b07-a4b0-557b5e890f97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:24.901738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af70ed1-1125-4b07-a4b0-557b5e890f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:24.903333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af70ed1-1125-4b07-a4b0-557b5e890f97 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:24.913955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1163/2310] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:31.715877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefafc1-ce6c-4f88-a765-3c4123bb3d4d service nova] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:31.717811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-bdefafc1-ce6c-4f88-a765-3c4123bb3d4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:31.790238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefafc1-ce6c-4f88-a765-3c4123bb3d4d service nova] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:31.795348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:31.795987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:31.796255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:31.796821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] PATH_INFO: `/v3/users/8dd19844068e4ce9a35235db8deb0044` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:31.797646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd19844068e4ce9a35235db8deb0044)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:31.799607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:31.851663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd19844068e4ce9a35235db8deb0044'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:31.852424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd19844068e4ce9a35235db8deb0044'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:31.865648 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd19844068e4ce9a35235db8deb0044'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:31.872983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd19844068e4ce9a35235db8deb0044'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:31.873722 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd19844068e4ce9a35235db8deb0044'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:31.886345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1148/2311] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:49:31 2022] DELETE /identity/v3/users/8dd19844068e4ce9a35235db8deb0044 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:49:31.891193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6ac01-3198-43e5-8fc9-9dafbdc8184a None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:31.892041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6ac01-3198-43e5-8fc9-9dafbdc8184a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:31.892227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6ac01-3198-43e5-8fc9-9dafbdc8184a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:31.911666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-41a6ac01-3198-43e5-8fc9-9dafbdc8184a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:31.912961 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-41a6ac01-3198-43e5-8fc9-9dafbdc8184a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:31.970851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1164/2312] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:31.980314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:31.981769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:32.038741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee172c-456a-475a-95d9-6c0168fb9d91 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:32.042592 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b92cfa-0b2e-40e9-9721-ee867120c578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:32.043204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b92cfa-0b2e-40e9-9721-ee867120c578 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:32.043483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b92cfa-0b2e-40e9-9721-ee867120c578 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:32.043752 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b92cfa-0b2e-40e9-9721-ee867120c578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:32.053905 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b92cfa-0b2e-40e9-9721-ee867120c578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:32.055326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b92cfa-0b2e-40e9-9721-ee867120c578 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:32.064695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1149/2313] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:49:32.336050 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6ac01-3198-43e5-8fc9-9dafbdc8184a None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:32.337522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-41a6ac01-3198-43e5-8fc9-9dafbdc8184a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:32.351744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6ac01-3198-43e5-8fc9-9dafbdc8184a None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:32.356397 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:32.357021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:32.357297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:32.357569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] PATH_INFO: `/v3/projects/6743f1434e0c44c58b66c23758c6b26c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:32.358743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6743f1434e0c44c58b66c23758c6b26c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:32.360684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:32.405190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:32.408415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6743f1434e0c44c58b66c23758c6b26c', 'request_id': 'req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:32.414715 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6743f1434e0c44c58b66c23758c6b26c', 'request_id': 'req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:32.420244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6743f1434e0c44c58b66c23758c6b26c', 'request_id': 'req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:32.432293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] The token cache is being invalidate because project 6743f1434e0c44c58b66c23758c6b26c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:49:32.432706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:32.433372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:49:32.434885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1165/2314] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:49:32 2022] DELETE /identity/v3/projects/6743f1434e0c44c58b66c23758c6b26c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:49:40.724386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595efeca-c77a-4732-97bd-c9a4e8e6562c None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:40.724928 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595efeca-c77a-4732-97bd-c9a4e8e6562c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:40.724928 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595efeca-c77a-4732-97bd-c9a4e8e6562c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:40.745192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-595efeca-c77a-4732-97bd-c9a4e8e6562c None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:40.746470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-595efeca-c77a-4732-97bd-c9a4e8e6562c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:40.764456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:40.807186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1150/2315] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:49:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:49:40.852368 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:40.853850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:40.935354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a116c0e-1e4a-4783-bbd7-4e5bbd46071a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:40.941937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d09be-5b69-4a0e-b4ca-3a31a3a3eba5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:40.942486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d09be-5b69-4a0e-b4ca-3a31a3a3eba5 service swift] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:40.942769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d09be-5b69-4a0e-b4ca-3a31a3a3eba5 service swift] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:40.943043 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d09be-5b69-4a0e-b4ca-3a31a3a3eba5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:40.955442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d09be-5b69-4a0e-b4ca-3a31a3a3eba5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:40.957305 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d09be-5b69-4a0e-b4ca-3a31a3a3eba5 service swift] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:40.966650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1166/2316] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:49:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:49:41.825728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577110d-3ac0-40d0-a63d-8b5e3c27eb27 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:41.826222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577110d-3ac0-40d0-a63d-8b5e3c27eb27 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:41.826519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577110d-3ac0-40d0-a63d-8b5e3c27eb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:41.846432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d577110d-3ac0-40d0-a63d-8b5e3c27eb27 None None] MFA Rules not processed for user `3c0b7528b0ff4b09923a0e951d052ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:41.847738 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d577110d-3ac0-40d0-a63d-8b5e3c27eb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:41.917304 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1151/2317] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:49:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:49:50.962758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:DELETE=4,keystone:SELECT=39,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:51.903329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=40 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:49:52.026822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d4dd0-fab1-4c4e-869f-b225f5e460f7 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:52.027330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d4dd0-fab1-4c4e-869f-b225f5e460f7 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:52.027450 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d4dd0-fab1-4c4e-869f-b225f5e460f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:52.047841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-108d4dd0-fab1-4c4e-869f-b225f5e460f7 None None] MFA Rules not processed for user `f928852d90b94ac5979e4a7806896b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:52.049097 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-108d4dd0-fab1-4c4e-869f-b225f5e460f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:52.160097 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1167/2318] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:49:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:58.517637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278d2a2-2218-4c77-8c47-9b463f3d8722 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:58.518205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278d2a2-2218-4c77-8c47-9b463f3d8722 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:58.518456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278d2a2-2218-4c77-8c47-9b463f3d8722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:58.539418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a278d2a2-2218-4c77-8c47-9b463f3d8722 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:49:58.540648 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a278d2a2-2218-4c77-8c47-9b463f3d8722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:49:58.547864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1152/2319] 10.222.0.26 () {58 vars in 967 bytes} [Tue Oct 4 08:49:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:49:58.561135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d4dd0-fab1-4c4e-869f-b225f5e460f7 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:49:58.572935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d4dd0-fab1-4c4e-869f-b225f5e460f7 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:49:58.579378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a2310-7601-456f-a8fd-50a489488e44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:49:58.579992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a2310-7601-456f-a8fd-50a489488e44 service swift] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:49:58.580277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a2310-7601-456f-a8fd-50a489488e44 service swift] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:49:58.580553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a2310-7601-456f-a8fd-50a489488e44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:49:58.589169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a2310-7601-456f-a8fd-50a489488e44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:49:58.591859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a2310-7601-456f-a8fd-50a489488e44 service swift] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:49:58.600553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1168/2320] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:49:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:50:00.739700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54422f7f-cfa0-4a38-813b-01f969bc872a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:00.740212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54422f7f-cfa0-4a38-813b-01f969bc872a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:00.740329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54422f7f-cfa0-4a38-813b-01f969bc872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:00.759092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-54422f7f-cfa0-4a38-813b-01f969bc872a None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:00.760282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-54422f7f-cfa0-4a38-813b-01f969bc872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:00.767884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1153/2321] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:50:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:00.780802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a2310-7601-456f-a8fd-50a489488e44 service swift] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:00.791436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a2310-7601-456f-a8fd-50a489488e44 service swift] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:00.796637 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77424dc-a462-41d2-be15-b9e948e06786 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:00.797304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77424dc-a462-41d2-be15-b9e948e06786 service swift] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:00.797683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77424dc-a462-41d2-be15-b9e948e06786 service swift] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:00.798701 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77424dc-a462-41d2-be15-b9e948e06786 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:00.807035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77424dc-a462-41d2-be15-b9e948e06786 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:00.808864 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77424dc-a462-41d2-be15-b9e948e06786 service swift] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:00.818099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1169/2322] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:50:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:50:01.243516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54422f7f-cfa0-4a38-813b-01f969bc872a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:01.245068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-54422f7f-cfa0-4a38-813b-01f969bc872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:01.397363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54422f7f-cfa0-4a38-813b-01f969bc872a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:01.406622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb547755-6bd2-4a57-a896-a3eab2ec7feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:01.409950 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb547755-6bd2-4a57-a896-a3eab2ec7feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:01.410315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb547755-6bd2-4a57-a896-a3eab2ec7feb admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:01.410688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb547755-6bd2-4a57-a896-a3eab2ec7feb admin admin] PATH_INFO: `/v3/users/efea00dacdca4d1d8e919fb446e164ea` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:01.412883 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb547755-6bd2-4a57-a896-a3eab2ec7feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=efea00dacdca4d1d8e919fb446e164ea)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:01.416203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb547755-6bd2-4a57-a896-a3eab2ec7feb admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:01.490683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-bb547755-6bd2-4a57-a896-a3eab2ec7feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efea00dacdca4d1d8e919fb446e164ea'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:01.491583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-bb547755-6bd2-4a57-a896-a3eab2ec7feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efea00dacdca4d1d8e919fb446e164ea'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:01.507305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-bb547755-6bd2-4a57-a896-a3eab2ec7feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efea00dacdca4d1d8e919fb446e164ea'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:01.517623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-bb547755-6bd2-4a57-a896-a3eab2ec7feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efea00dacdca4d1d8e919fb446e164ea'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:01.518492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-bb547755-6bd2-4a57-a896-a3eab2ec7feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efea00dacdca4d1d8e919fb446e164ea'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:01.541111 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1154/2323] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:50:01 2022] DELETE /identity/v3/users/efea00dacdca4d1d8e919fb446e164ea => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:50:01.774335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77424dc-a462-41d2-be15-b9e948e06786 service swift] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:01.789231 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77424dc-a462-41d2-be15-b9e948e06786 service swift] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:01.868912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e00384-272a-465f-a7eb-c082a81d8cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:01.869705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e00384-272a-465f-a7eb-c082a81d8cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:01.870244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e00384-272a-465f-a7eb-c082a81d8cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:01.870727 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e00384-272a-465f-a7eb-c082a81d8cb7 admin admin] PATH_INFO: `/v3/projects/47f943270eaa45a4b5aec895741d553d` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:01.871975 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e00384-272a-465f-a7eb-c082a81d8cb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47f943270eaa45a4b5aec895741d553d)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:01.875265 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e00384-272a-465f-a7eb-c082a81d8cb7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:01.957269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e6e00384-272a-465f-a7eb-c082a81d8cb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47f943270eaa45a4b5aec895741d553d', 'request_id': 'req-e6e00384-272a-465f-a7eb-c082a81d8cb7'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:01.974459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e6e00384-272a-465f-a7eb-c082a81d8cb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47f943270eaa45a4b5aec895741d553d', 'request_id': 'req-e6e00384-272a-465f-a7eb-c082a81d8cb7'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:01.993086 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e6e00384-272a-465f-a7eb-c082a81d8cb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47f943270eaa45a4b5aec895741d553d', 'request_id': 'req-e6e00384-272a-465f-a7eb-c082a81d8cb7'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:02.017901 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e6e00384-272a-465f-a7eb-c082a81d8cb7 admin admin] The token cache is being invalidate because project 47f943270eaa45a4b5aec895741d553d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:50:02.018195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e6e00384-272a-465f-a7eb-c082a81d8cb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:02.020551 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e6e00384-272a-465f-a7eb-c082a81d8cb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:02.022631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1170/2324] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:50:01 2022] DELETE /identity/v3/projects/47f943270eaa45a4b5aec895741d553d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:50:02.027570 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169057eb-1394-4e38-8dac-f433ff0972f4 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:02.028071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169057eb-1394-4e38-8dac-f433ff0972f4 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:02.028188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169057eb-1394-4e38-8dac-f433ff0972f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:02.056649 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-169057eb-1394-4e38-8dac-f433ff0972f4 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:02.057986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-169057eb-1394-4e38-8dac-f433ff0972f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:02.180871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1155/2325] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:02.272526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f260a-2092-4475-9144-9c92303cb3a8 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:02.273199 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f260a-2092-4475-9144-9c92303cb3a8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:02.273681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f260a-2092-4475-9144-9c92303cb3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:02.296024 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-b26f260a-2092-4475-9144-9c92303cb3a8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:02.297641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b26f260a-2092-4475-9144-9c92303cb3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:02.312761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1171/2326] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:02.313247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169057eb-1394-4e38-8dac-f433ff0972f4 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:02.328586 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169057eb-1394-4e38-8dac-f433ff0972f4 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:02.342298 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e39058-addf-4b18-b8b7-1e90a4160c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:02.343638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e39058-addf-4b18-b8b7-1e90a4160c1e admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:02.343967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e39058-addf-4b18-b8b7-1e90a4160c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:02.344249 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e39058-addf-4b18-b8b7-1e90a4160c1e admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:02.345895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e39058-addf-4b18-b8b7-1e90a4160c1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:50:02.346347 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e39058-addf-4b18-b8b7-1e90a4160c1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:02.348440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e39058-addf-4b18-b8b7-1e90a4160c1e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:02.351333 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:02.367774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1156/2327] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:50:02.372289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f260a-2092-4475-9144-9c92303cb3a8 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:02.376804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:02.385391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f260a-2092-4475-9144-9c92303cb3a8 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:02.392641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87b3d8-cfc5-4bd0-b990-56a0997aff27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:02.393615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87b3d8-cfc5-4bd0-b990-56a0997aff27 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:02.393615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87b3d8-cfc5-4bd0-b990-56a0997aff27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:02.393876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b87b3d8-cfc5-4bd0-b990-56a0997aff27 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:02.394472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87b3d8-cfc5-4bd0-b990-56a0997aff27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:02.396360 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b87b3d8-cfc5-4bd0-b990-56a0997aff27 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:02.433643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1172/2328] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:50:02 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:50:02.434638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e39058-addf-4b18-b8b7-1e90a4160c1e admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:02.454182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e39058-addf-4b18-b8b7-1e90a4160c1e admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:02.567015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ec5f6-27f9-4bae-a345-f6f359cc216c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:02.567813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ec5f6-27f9-4bae-a345-f6f359cc216c admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:02.568612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ec5f6-27f9-4bae-a345-f6f359cc216c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:02.568612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ec5f6-27f9-4bae-a345-f6f359cc216c admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:02.569404 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ec5f6-27f9-4bae-a345-f6f359cc216c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:02.580660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ec5f6-27f9-4bae-a345-f6f359cc216c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:02.630697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87b3d8-cfc5-4bd0-b990-56a0997aff27 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:02.632336 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1157/2329] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:50:02 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:50:02.646895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b87b3d8-cfc5-4bd0-b990-56a0997aff27 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:02.669893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d94a11-b345-4be2-9acf-414f5d696e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:02.670703 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d94a11-b345-4be2-9acf-414f5d696e9a admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:02.671126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d94a11-b345-4be2-9acf-414f5d696e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:02.671495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d94a11-b345-4be2-9acf-414f5d696e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:02.672417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d94a11-b345-4be2-9acf-414f5d696e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:02.675394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d94a11-b345-4be2-9acf-414f5d696e9a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:02.698980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ec5f6-27f9-4bae-a345-f6f359cc216c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:02.713401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1173/2330] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:50:02.713641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ec5f6-27f9-4bae-a345-f6f359cc216c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:02.720954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38748a-aaf2-4df1-b502-15e9d71ffbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:02.721813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38748a-aaf2-4df1-b502-15e9d71ffbdc admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:02.722191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38748a-aaf2-4df1-b502-15e9d71ffbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:02.722559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38748a-aaf2-4df1-b502-15e9d71ffbdc admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:02.723436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38748a-aaf2-4df1-b502-15e9d71ffbdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:02.725989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38748a-aaf2-4df1-b502-15e9d71ffbdc admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:02.750240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1158/2331] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:50:02.754292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d94a11-b345-4be2-9acf-414f5d696e9a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:02.770362 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d94a11-b345-4be2-9acf-414f5d696e9a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:02.776283 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4519e-5581-44d5-be0e-c33344942d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:02.776962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4519e-5581-44d5-be0e-c33344942d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:02.777240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4519e-5581-44d5-be0e-c33344942d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:02.777513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4519e-5581-44d5-be0e-c33344942d32 admin admin] PATH_INFO: `/v3/projects/db489a9ac5354d6f8f31c88cf20c34fd/users/db83f1e6aebf487d88fe0adaeacf306c/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:02.805733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4519e-5581-44d5-be0e-c33344942d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db489a9ac5354d6f8f31c88cf20c34fd, user_id=db83f1e6aebf487d88fe0adaeacf306c, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:02.809034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4519e-5581-44d5-be0e-c33344942d32 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:02.857465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1174/2332] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:50:02 2022] PUT /identity/v3/projects/db489a9ac5354d6f8f31c88cf20c34fd/users/db83f1e6aebf487d88fe0adaeacf306c/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:50:02.868816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636985f-9062-4238-b761-d6470699a815 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:02.869367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636985f-9062-4238-b761-d6470699a815 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:02.869636 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636985f-9062-4238-b761-d6470699a815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:02.933093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-0636985f-9062-4238-b761-d6470699a815 None None] MFA Rules not processed for user `db83f1e6aebf487d88fe0adaeacf306c`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:02.936001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0636985f-9062-4238-b761-d6470699a815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:03.060697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4519e-5581-44d5-be0e-c33344942d32 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:03.065773 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7cc4519e-5581-44d5-be0e-c33344942d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:03.074797 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1159/2333] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:50:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:50:03.151233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4519e-5581-44d5-be0e-c33344942d32 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:03.156320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9a753-31f2-440a-9e1c-2cb9b9890ad8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '913a866ddc074b7f9aa142c70da98a69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:03.157051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9a753-31f2-440a-9e1c-2cb9b9890ad8 service glance] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:03.157427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9a753-31f2-440a-9e1c-2cb9b9890ad8 service glance] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:03.157829 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9a753-31f2-440a-9e1c-2cb9b9890ad8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:03.167205 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9a753-31f2-440a-9e1c-2cb9b9890ad8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:03.171546 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9a753-31f2-440a-9e1c-2cb9b9890ad8 service glance] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:03.190953 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1175/2334] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:50:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:50:13.040792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:13.179721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=49,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:17.446910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636985f-9062-4238-b761-d6470699a815 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:17.448869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0636985f-9062-4238-b761-d6470699a815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:17.507355 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636985f-9062-4238-b761-d6470699a815 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:17.510943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab2e67-b251-42f2-99ff-20c8b5c620f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:17.511553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab2e67-b251-42f2-99ff-20c8b5c620f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:17.511834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab2e67-b251-42f2-99ff-20c8b5c620f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:17.512105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab2e67-b251-42f2-99ff-20c8b5c620f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:17.513833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ebab2e67-b251-42f2-99ff-20c8b5c620f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:17.577324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab2e67-b251-42f2-99ff-20c8b5c620f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:17.578796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab2e67-b251-42f2-99ff-20c8b5c620f7 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:17.588267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1160/2335] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:50:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:50:18.625196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275a726-dabf-41ac-8b1a-03101ceb8a28 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:18.625768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275a726-dabf-41ac-8b1a-03101ceb8a28 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:18.625768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275a726-dabf-41ac-8b1a-03101ceb8a28 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:18.627094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1176/2336] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:50:18.633096 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d1b84-8e3b-4200-87ca-6c46c9c897a2 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:18.633582 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d1b84-8e3b-4200-87ca-6c46c9c897a2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:18.633864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d1b84-8e3b-4200-87ca-6c46c9c897a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:18.662003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-eb9d1b84-8e3b-4200-87ca-6c46c9c897a2 None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:18.665624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-eb9d1b84-8e3b-4200-87ca-6c46c9c897a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:18.760554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1161/2337] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:50:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:18.776556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275a726-dabf-41ac-8b1a-03101ceb8a28 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:18.780530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8275a726-dabf-41ac-8b1a-03101ceb8a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:18.793770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275a726-dabf-41ac-8b1a-03101ceb8a28 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:18.797328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6212a25-4082-4b70-87df-7ee76e0c788a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '913a866ddc074b7f9aa142c70da98a69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:18.797966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6212a25-4082-4b70-87df-7ee76e0c788a service glance] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:18.798241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6212a25-4082-4b70-87df-7ee76e0c788a service glance] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:18.798510 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6212a25-4082-4b70-87df-7ee76e0c788a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:18.808321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6212a25-4082-4b70-87df-7ee76e0c788a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:18.809816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6212a25-4082-4b70-87df-7ee76e0c788a service glance] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:18.820700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1177/2338] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:50:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:50:18.891389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418eee6-0a0c-45a0-8c04-7d82d65cd24d None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:18.891698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418eee6-0a0c-45a0-8c04-7d82d65cd24d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:18.891994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418eee6-0a0c-45a0-8c04-7d82d65cd24d None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:18.893356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1162/2339] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 08:50:18.898438 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4cf1f-aabd-4cce-9feb-34e676f80e48 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:18.898767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4cf1f-aabd-4cce-9feb-34e676f80e48 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:18.899175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4cf1f-aabd-4cce-9feb-34e676f80e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:18.918931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-83f4cf1f-aabd-4cce-9feb-34e676f80e48 None None] MFA Rules not processed for user `50d3b75cbc5c4d1686f4d93248e04746`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:18.920261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-83f4cf1f-aabd-4cce-9feb-34e676f80e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:18.927431 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1178/2340] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:50:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:50:18.940192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418eee6-0a0c-45a0-8c04-7d82d65cd24d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:18.941906 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d418eee6-0a0c-45a0-8c04-7d82d65cd24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:18.955612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418eee6-0a0c-45a0-8c04-7d82d65cd24d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:18.959172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf498e-7f2c-403c-a0e5-4c55fdaeb055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:18.959765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf498e-7f2c-403c-a0e5-4c55fdaeb055 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:18.960068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf498e-7f2c-403c-a0e5-4c55fdaeb055 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:18.960335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf498e-7f2c-403c-a0e5-4c55fdaeb055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:18.968114 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf498e-7f2c-403c-a0e5-4c55fdaeb055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:18.969581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf498e-7f2c-403c-a0e5-4c55fdaeb055 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:18.977888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1163/2341] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:50:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:50:19.068158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b2a4d-dce9-4800-aee7-c8d05a32e028 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:19.068564 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b2a4d-dce9-4800-aee7-c8d05a32e028 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:19.068906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b2a4d-dce9-4800-aee7-c8d05a32e028 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:19.070404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1179/2342] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:50:19.076807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e042c-07e6-4874-bde2-6ce9a9fc52b4 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:19.077180 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e042c-07e6-4874-bde2-6ce9a9fc52b4 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:19.077526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e042c-07e6-4874-bde2-6ce9a9fc52b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:19.098737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-044e042c-07e6-4874-bde2-6ce9a9fc52b4 None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:19.100284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-044e042c-07e6-4874-bde2-6ce9a9fc52b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:19.110122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1164/2343] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:50:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:19.121425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b2a4d-dce9-4800-aee7-c8d05a32e028 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:19.133460 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b2a4d-dce9-4800-aee7-c8d05a32e028 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:19.137827 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b94a4d-0bd6-46f0-b66b-4ab13a6592c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:19.139080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b94a4d-0bd6-46f0-b66b-4ab13a6592c2 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:19.139568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b94a4d-0bd6-46f0-b66b-4ab13a6592c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:19.139843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b94a4d-0bd6-46f0-b66b-4ab13a6592c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:19.151122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b94a4d-0bd6-46f0-b66b-4ab13a6592c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:19.152624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b94a4d-0bd6-46f0-b66b-4ab13a6592c2 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:19.164586 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1180/2344] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:50:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:50:22.353948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fed11d-078c-4fa0-adae-ec07f5429bd8 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:22.354465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fed11d-078c-4fa0-adae-ec07f5429bd8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:22.354650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fed11d-078c-4fa0-adae-ec07f5429bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:22.376804 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-37fed11d-078c-4fa0-adae-ec07f5429bd8 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:22.378117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-37fed11d-078c-4fa0-adae-ec07f5429bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:22.395267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:22.435495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1165/2345] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:50:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:22.447196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b94a4d-0bd6-46f0-b66b-4ab13a6592c2 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:22.448578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-83b94a4d-0bd6-46f0-b66b-4ab13a6592c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:22.515153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b94a4d-0bd6-46f0-b66b-4ab13a6592c2 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:22.520096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e94e99-4e1c-415a-8e63-b371c9d556ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:22.520687 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e94e99-4e1c-415a-8e63-b371c9d556ce service swift] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:22.520965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e94e99-4e1c-415a-8e63-b371c9d556ce service swift] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:22.521248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e94e99-4e1c-415a-8e63-b371c9d556ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:22.529064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e94e99-4e1c-415a-8e63-b371c9d556ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:22.530670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e94e99-4e1c-415a-8e63-b371c9d556ce service swift] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:22.538645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1181/2346] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:50:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:50:32.427954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=15 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:32.535333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=47 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:37.856030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fed11d-078c-4fa0-adae-ec07f5429bd8 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:37.865279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fed11d-078c-4fa0-adae-ec07f5429bd8 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:37.868959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46480075-a094-447d-bd6f-ebca0d7bc74b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:37.869570 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46480075-a094-447d-bd6f-ebca0d7bc74b service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:37.869896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46480075-a094-447d-bd6f-ebca0d7bc74b service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:37.870168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46480075-a094-447d-bd6f-ebca0d7bc74b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:37.871927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-46480075-a094-447d-bd6f-ebca0d7bc74b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:37.885602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46480075-a094-447d-bd6f-ebca0d7bc74b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:37.887123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46480075-a094-447d-bd6f-ebca0d7bc74b service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:37.898506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1166/2347] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:50:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:50:39.077679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e94e99-4e1c-415a-8e63-b371c9d556ce service swift] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:39.086839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e94e99-4e1c-415a-8e63-b371c9d556ce service swift] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:39.090779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e148944-b839-4ce9-a02b-3cd5977910e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:39.092004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e148944-b839-4ce9-a02b-3cd5977910e0 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:39.092284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e148944-b839-4ce9-a02b-3cd5977910e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:39.092556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e148944-b839-4ce9-a02b-3cd5977910e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:39.094312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2e148944-b839-4ce9-a02b-3cd5977910e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:39.107203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e148944-b839-4ce9-a02b-3cd5977910e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:39.108629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e148944-b839-4ce9-a02b-3cd5977910e0 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:39.119267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1182/2348] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:50:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:50:45.115662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46480075-a094-447d-bd6f-ebca0d7bc74b service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:45.176589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46480075-a094-447d-bd6f-ebca0d7bc74b service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:45.181099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:45.181741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:45.182023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:45.182296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] PATH_INFO: `/v3/users/db83f1e6aebf487d88fe0adaeacf306c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:45.183399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=db83f1e6aebf487d88fe0adaeacf306c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:45.185317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:45.229428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db83f1e6aebf487d88fe0adaeacf306c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:45.230018 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db83f1e6aebf487d88fe0adaeacf306c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:45.242165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db83f1e6aebf487d88fe0adaeacf306c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:45.248775 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db83f1e6aebf487d88fe0adaeacf306c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:45.249407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db83f1e6aebf487d88fe0adaeacf306c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:45.262185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1167/2349] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:50:45 2022] DELETE /identity/v3/users/db83f1e6aebf487d88fe0adaeacf306c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:50:45.266723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc01c8-2f03-495a-857b-3024627d4de7 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:45.267209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc01c8-2f03-495a-857b-3024627d4de7 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:45.267310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc01c8-2f03-495a-857b-3024627d4de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:45.286638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-8ddc01c8-2f03-495a-857b-3024627d4de7 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:45.287841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8ddc01c8-2f03-495a-857b-3024627d4de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:45.345457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1183/2350] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:45.355389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:45.356813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:45.412940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e1a7c-cde2-48e9-ae11-d8bfdb7389bb admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:45.416540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de7c86-38dd-4408-8eac-c1d58221f39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:45.417133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de7c86-38dd-4408-8eac-c1d58221f39f service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:45.417410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de7c86-38dd-4408-8eac-c1d58221f39f service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:45.417683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de7c86-38dd-4408-8eac-c1d58221f39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:45.427129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de7c86-38dd-4408-8eac-c1d58221f39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:45.428550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de7c86-38dd-4408-8eac-c1d58221f39f service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:45.437737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1168/2351] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:50:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:50:45.697570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc01c8-2f03-495a-857b-3024627d4de7 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:45.699016 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8ddc01c8-2f03-495a-857b-3024627d4de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:45.710988 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc01c8-2f03-495a-857b-3024627d4de7 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:45.715853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:45.716498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:45.716777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:45.717052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a admin admin] PATH_INFO: `/v3/projects/db489a9ac5354d6f8f31c88cf20c34fd` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:45.718315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a admin admin] RBAC: Authorizing `identity:delete_project(project_id=db489a9ac5354d6f8f31c88cf20c34fd)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:45.720242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:45.766821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db489a9ac5354d6f8f31c88cf20c34fd', 'request_id': 'req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:45.772959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db489a9ac5354d6f8f31c88cf20c34fd', 'request_id': 'req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:45.778548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db489a9ac5354d6f8f31c88cf20c34fd', 'request_id': 'req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:45.791876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a admin admin] The token cache is being invalidate because project db489a9ac5354d6f8f31c88cf20c34fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:50:45.792160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:45.792770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-c4af9dfa-d67c-4f31-8f42-f14cc692e46a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:50:45.794272 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1184/2352] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:50:45 2022] DELETE /identity/v3/projects/db489a9ac5354d6f8f31c88cf20c34fd => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:50:45.799828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12091924-db91-4086-b77f-bb99d9174b43 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:45.800113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12091924-db91-4086-b77f-bb99d9174b43 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:45.800413 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12091924-db91-4086-b77f-bb99d9174b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:45.820547 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-12091924-db91-4086-b77f-bb99d9174b43 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:45.821753 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-12091924-db91-4086-b77f-bb99d9174b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:45.826062 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:45.881873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1169/2353] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:45.890862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16743663-916d-40d9-b2a5-b2626718c9d5 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:45.891142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16743663-916d-40d9-b2a5-b2626718c9d5 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:45.891411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16743663-916d-40d9-b2a5-b2626718c9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:45.911008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-16743663-916d-40d9-b2a5-b2626718c9d5 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:45.912193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-16743663-916d-40d9-b2a5-b2626718c9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:45.920459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1185/2354] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:45.924479 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12091924-db91-4086-b77f-bb99d9174b43 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:45.934066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12091924-db91-4086-b77f-bb99d9174b43 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:45.938589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e4add-f14b-4e37-b6ed-bc102c6d91f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:45.939181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e4add-f14b-4e37-b6ed-bc102c6d91f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:45.939460 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e4add-f14b-4e37-b6ed-bc102c6d91f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:45.939734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e4add-f14b-4e37-b6ed-bc102c6d91f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:45.940366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e4add-f14b-4e37-b6ed-bc102c6d91f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:50:45.940669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e4add-f14b-4e37-b6ed-bc102c6d91f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:45.942299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e4add-f14b-4e37-b6ed-bc102c6d91f7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:45.953569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1170/2355] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:50:45.957601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16743663-916d-40d9-b2a5-b2626718c9d5 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:45.967273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16743663-916d-40d9-b2a5-b2626718c9d5 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:45.971867 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fe705-414b-49e2-ac34-18ed1d9d1db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:45.972446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fe705-414b-49e2-ac34-18ed1d9d1db7 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:45.972724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fe705-414b-49e2-ac34-18ed1d9d1db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:45.972994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690fe705-414b-49e2-ac34-18ed1d9d1db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:45.973620 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690fe705-414b-49e2-ac34-18ed1d9d1db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:45.975529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690fe705-414b-49e2-ac34-18ed1d9d1db7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:46.001930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1186/2356] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:50:45 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:50:46.006117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e4add-f14b-4e37-b6ed-bc102c6d91f7 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:46.015849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e4add-f14b-4e37-b6ed-bc102c6d91f7 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:46.075462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29598bca-c7cb-4c3a-8285-eae175053114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:46.076016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29598bca-c7cb-4c3a-8285-eae175053114 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:46.076296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29598bca-c7cb-4c3a-8285-eae175053114 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:46.076570 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29598bca-c7cb-4c3a-8285-eae175053114 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:46.077196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29598bca-c7cb-4c3a-8285-eae175053114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:46.079151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29598bca-c7cb-4c3a-8285-eae175053114 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:46.100164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1171/2357] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:50:46 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:50:46.104492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fe705-414b-49e2-ac34-18ed1d9d1db7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:46.115352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690fe705-414b-49e2-ac34-18ed1d9d1db7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:46.119958 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027e9ae-6786-46de-bb17-914ccc2bee1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:46.120547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027e9ae-6786-46de-bb17-914ccc2bee1e admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:46.120825 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027e9ae-6786-46de-bb17-914ccc2bee1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:46.121094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027e9ae-6786-46de-bb17-914ccc2bee1e admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:46.121763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027e9ae-6786-46de-bb17-914ccc2bee1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:46.123514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027e9ae-6786-46de-bb17-914ccc2bee1e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:46.136636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1187/2358] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:50:46.137778 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29598bca-c7cb-4c3a-8285-eae175053114 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:46.147637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29598bca-c7cb-4c3a-8285-eae175053114 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:46.152378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc250ab-9297-4ea0-8521-498a53ac0f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:46.153591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc250ab-9297-4ea0-8521-498a53ac0f7c admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:46.153591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc250ab-9297-4ea0-8521-498a53ac0f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:46.153591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc250ab-9297-4ea0-8521-498a53ac0f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:46.155203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc250ab-9297-4ea0-8521-498a53ac0f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:46.156296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc250ab-9297-4ea0-8521-498a53ac0f7c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:46.167282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1172/2359] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:50:46.172113 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027e9ae-6786-46de-bb17-914ccc2bee1e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:46.182222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027e9ae-6786-46de-bb17-914ccc2bee1e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:46.189369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b7efc-8293-42df-9217-bacc19e63bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:46.190055 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b7efc-8293-42df-9217-bacc19e63bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:46.190337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b7efc-8293-42df-9217-bacc19e63bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:46.190607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b7efc-8293-42df-9217-bacc19e63bfb admin admin] PATH_INFO: `/v3/projects/4a75a59f42a74dee854ddc7b5f109143/users/dfe240ebc7c643358db4ae3a9ef132ea/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:46.206028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b7efc-8293-42df-9217-bacc19e63bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a75a59f42a74dee854ddc7b5f109143, user_id=dfe240ebc7c643358db4ae3a9ef132ea, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:46.208697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b7efc-8293-42df-9217-bacc19e63bfb admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:46.223678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1188/2360] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:50:46 2022] PUT /identity/v3/projects/4a75a59f42a74dee854ddc7b5f109143/users/dfe240ebc7c643358db4ae3a9ef132ea/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:50:46.228626 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e839ea0-861f-423d-91ba-27dd713681fe None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:46.228981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e839ea0-861f-423d-91ba-27dd713681fe None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:46.229427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e839ea0-861f-423d-91ba-27dd713681fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:46.250558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-3e839ea0-861f-423d-91ba-27dd713681fe None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:46.251775 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3e839ea0-861f-423d-91ba-27dd713681fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:46.314698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1173/2361] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:46.324284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b7efc-8293-42df-9217-bacc19e63bfb admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:46.326752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-aa0b7efc-8293-42df-9217-bacc19e63bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:46.336476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:46.384775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b7efc-8293-42df-9217-bacc19e63bfb admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:46.388898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d5c2a-fb38-4ec1-abfe-af25dc317b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:46.389496 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d5c2a-fb38-4ec1-abfe-af25dc317b31 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:46.389800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d5c2a-fb38-4ec1-abfe-af25dc317b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:46.390070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d5c2a-fb38-4ec1-abfe-af25dc317b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:46.398945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d5c2a-fb38-4ec1-abfe-af25dc317b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:46.400226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d5c2a-fb38-4ec1-abfe-af25dc317b31 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:46.410478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1189/2362] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:50:50.582192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839dcfc-8b69-4459-8963-b451defa73e2 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:50.582686 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839dcfc-8b69-4459-8963-b451defa73e2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:50.582794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839dcfc-8b69-4459-8963-b451defa73e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:50.630662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d839dcfc-8b69-4459-8963-b451defa73e2 None None] MFA Rules not processed for user `dfe240ebc7c643358db4ae3a9ef132ea`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:50.631880 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d839dcfc-8b69-4459-8963-b451defa73e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:50.635756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:50.709975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1174/2363] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:50:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:50.714747 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d5c2a-fb38-4ec1-abfe-af25dc317b31 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:50.725502 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d5c2a-fb38-4ec1-abfe-af25dc317b31 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:50.730217 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbf4c2-27ac-4512-93b9-4ed157f3f598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:50.730835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbf4c2-27ac-4512-93b9-4ed157f3f598 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:50.731316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbf4c2-27ac-4512-93b9-4ed157f3f598 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:50.731596 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbf4c2-27ac-4512-93b9-4ed157f3f598 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:50.732292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bbf4c2-27ac-4512-93b9-4ed157f3f598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:50.734264 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bbf4c2-27ac-4512-93b9-4ed157f3f598 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:50.762764 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1190/2364] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:50:50 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:50:50.767389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839dcfc-8b69-4459-8963-b451defa73e2 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:50.778434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839dcfc-8b69-4459-8963-b451defa73e2 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:50.837740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17a7e8-043e-4aba-86b5-3219613aa34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:50.838367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17a7e8-043e-4aba-86b5-3219613aa34f admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:50.838656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17a7e8-043e-4aba-86b5-3219613aa34f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:50.838932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17a7e8-043e-4aba-86b5-3219613aa34f admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:50.839831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17a7e8-043e-4aba-86b5-3219613aa34f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:50.842414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17a7e8-043e-4aba-86b5-3219613aa34f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:50.867751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1175/2365] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:50:50 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:50:50.868989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbf4c2-27ac-4512-93b9-4ed157f3f598 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:50.879326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbf4c2-27ac-4512-93b9-4ed157f3f598 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:50.884222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b25e04-6fb1-45b4-b08e-badc03686a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:50.884771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b25e04-6fb1-45b4-b08e-badc03686a7c admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:50.885055 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b25e04-6fb1-45b4-b08e-badc03686a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:50.885347 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b25e04-6fb1-45b4-b08e-badc03686a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:50.886024 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b25e04-6fb1-45b4-b08e-badc03686a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:50.887638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b25e04-6fb1-45b4-b08e-badc03686a7c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:50.899378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1191/2366] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:50:50.903539 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17a7e8-043e-4aba-86b5-3219613aa34f admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:50.915050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17a7e8-043e-4aba-86b5-3219613aa34f admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:50.920219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7d4b3-398d-4f2b-a93c-f5ee89dd568e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:50.920902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7d4b3-398d-4f2b-a93c-f5ee89dd568e admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:50.921182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7d4b3-398d-4f2b-a93c-f5ee89dd568e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:50.921455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7d4b3-398d-4f2b-a93c-f5ee89dd568e admin admin] PATH_INFO: `/v3/projects/91af61bce2c14978bec974811300f333/users/06f3c8346a7b48a0a0b9631a9794f85d/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:50.938102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7d4b3-398d-4f2b-a93c-f5ee89dd568e admin admin] RBAC: Authorizing `identity:create_grant(project_id=91af61bce2c14978bec974811300f333, user_id=06f3c8346a7b48a0a0b9631a9794f85d, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:50.940669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7d4b3-398d-4f2b-a93c-f5ee89dd568e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:50.955806 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1176/2367] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:50:50 2022] PUT /identity/v3/projects/91af61bce2c14978bec974811300f333/users/06f3c8346a7b48a0a0b9631a9794f85d/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:50:50.959778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b25e04-6fb1-45b4-b08e-badc03686a7c admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:50.971225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b25e04-6fb1-45b4-b08e-badc03686a7c admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:51.029823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bacc27-6b91-43d2-9e4a-bc03e57ff777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:51.030437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bacc27-6b91-43d2-9e4a-bc03e57ff777 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:51.030716 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bacc27-6b91-43d2-9e4a-bc03e57ff777 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:51.030994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bacc27-6b91-43d2-9e4a-bc03e57ff777 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:51.031635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bacc27-6b91-43d2-9e4a-bc03e57ff777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:51.033242 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bacc27-6b91-43d2-9e4a-bc03e57ff777 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:51.043691 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1192/2368] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:50:51.047918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7d4b3-398d-4f2b-a93c-f5ee89dd568e admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:51.058461 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7d4b3-398d-4f2b-a93c-f5ee89dd568e admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:51.063190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03e6f4-343f-4067-8105-db5546d0e2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:51.063851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03e6f4-343f-4067-8105-db5546d0e2cd admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:51.064126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03e6f4-343f-4067-8105-db5546d0e2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:51.064396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03e6f4-343f-4067-8105-db5546d0e2cd admin admin] PATH_INFO: `/v3/projects/91af61bce2c14978bec974811300f333/users/06f3c8346a7b48a0a0b9631a9794f85d/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:51.066325 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03e6f4-343f-4067-8105-db5546d0e2cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=91af61bce2c14978bec974811300f333, user_id=06f3c8346a7b48a0a0b9631a9794f85d, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:51.068957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a03e6f4-343f-4067-8105-db5546d0e2cd admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:51.080888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1177/2369] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:50:51 2022] PUT /identity/v3/projects/91af61bce2c14978bec974811300f333/users/06f3c8346a7b48a0a0b9631a9794f85d/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:50:52.340153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bacc27-6b91-43d2-9e4a-bc03e57ff777 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:52.342107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-15bacc27-6b91-43d2-9e4a-bc03e57ff777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:52.456971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bacc27-6b91-43d2-9e4a-bc03e57ff777 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:52.470615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:52.559180 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ede1d3-91b1-4c48-bbf4-ad9a84d3296d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:52.560066 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ede1d3-91b1-4c48-bbf4-ad9a84d3296d service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:52.561342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ede1d3-91b1-4c48-bbf4-ad9a84d3296d service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:52.562046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ede1d3-91b1-4c48-bbf4-ad9a84d3296d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:52.564564 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-71ede1d3-91b1-4c48-bbf4-ad9a84d3296d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:52.686674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ede1d3-91b1-4c48-bbf4-ad9a84d3296d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:52.702834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ede1d3-91b1-4c48-bbf4-ad9a84d3296d service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:52.720202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1193/2370] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:50:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:50:55.807747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab6c92-32f8-428c-a1df-2994970963ab None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:55.808326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab6c92-32f8-428c-a1df-2994970963ab None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:55.808326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab6c92-32f8-428c-a1df-2994970963ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:55.837537 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a0ab6c92-32f8-428c-a1df-2994970963ab None None] MFA Rules not processed for user `06f3c8346a7b48a0a0b9631a9794f85d`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:55.838778 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a0ab6c92-32f8-428c-a1df-2994970963ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:55.885292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:50:55.915948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1178/2371] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:50:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:50:55.930133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ede1d3-91b1-4c48-bbf4-ad9a84d3296d service nova] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:50:55.931654 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-71ede1d3-91b1-4c48-bbf4-ad9a84d3296d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:55.946332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ede1d3-91b1-4c48-bbf4-ad9a84d3296d service nova] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:50:55.951983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d58262-2492-4722-8ae9-e639dd2d70bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:50:55.952597 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d58262-2492-4722-8ae9-e639dd2d70bc service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:55.953108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d58262-2492-4722-8ae9-e639dd2d70bc service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:55.953387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d58262-2492-4722-8ae9-e639dd2d70bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:56.011096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d58262-2492-4722-8ae9-e639dd2d70bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:50:56.012692 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d58262-2492-4722-8ae9-e639dd2d70bc service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:50:56.022314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1194/2372] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:50:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:50:57.241349 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fcd44-6aaf-46fe-b88f-b950ba4f1778 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:50:57.241900 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fcd44-6aaf-46fe-b88f-b950ba4f1778 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:50:57.241900 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fcd44-6aaf-46fe-b88f-b950ba4f1778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:50:57.263612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-5b7fcd44-6aaf-46fe-b88f-b950ba4f1778 None None] MFA Rules not processed for user `dfe240ebc7c643358db4ae3a9ef132ea`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:50:57.264831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5b7fcd44-6aaf-46fe-b88f-b950ba4f1778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:50:57.271941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1179/2373] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:50:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:05.588040 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427c548-420e-49ff-92d4-25e52de1ddb8 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:05.588559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427c548-420e-49ff-92d4-25e52de1ddb8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:05.588666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427c548-420e-49ff-92d4-25e52de1ddb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:05.608069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-f427c548-420e-49ff-92d4-25e52de1ddb8 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:05.609270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f427c548-420e-49ff-92d4-25e52de1ddb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:05.651788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:05.670683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1195/2374] 10.222.0.26 () {58 vars in 967 bytes} [Tue Oct 4 08:51:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:05.693380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fcd44-6aaf-46fe-b88f-b950ba4f1778 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:05.694850 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5b7fcd44-6aaf-46fe-b88f-b950ba4f1778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:05.765610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fcd44-6aaf-46fe-b88f-b950ba4f1778 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:05.770735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f250bf-a1b5-4cec-b7fe-63862d33b94e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:05.771347 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f250bf-a1b5-4cec-b7fe-63862d33b94e service swift] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:05.771622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f250bf-a1b5-4cec-b7fe-63862d33b94e service swift] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:05.771891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f250bf-a1b5-4cec-b7fe-63862d33b94e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:05.780501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f250bf-a1b5-4cec-b7fe-63862d33b94e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:05.782162 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f250bf-a1b5-4cec-b7fe-63862d33b94e service swift] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:05.790091 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1180/2375] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:51:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 08:51:13.619287 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427c548-420e-49ff-92d4-25e52de1ddb8 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:13.631788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427c548-420e-49ff-92d4-25e52de1ddb8 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:13.637252 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9efc92-c81c-4ab7-ad0c-b6e6eb27c207 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:13.637888 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9efc92-c81c-4ab7-ad0c-b6e6eb27c207 service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:13.638169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9efc92-c81c-4ab7-ad0c-b6e6eb27c207 service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:13.638447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9efc92-c81c-4ab7-ad0c-b6e6eb27c207 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:13.640164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1a9efc92-c81c-4ab7-ad0c-b6e6eb27c207 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:13.698405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9efc92-c81c-4ab7-ad0c-b6e6eb27c207 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:13.700021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9efc92-c81c-4ab7-ad0c-b6e6eb27c207 service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:13.708996 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1196/2376] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:15.786976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=43 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:22.827446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f250bf-a1b5-4cec-b7fe-63862d33b94e service swift] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:22.829040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-24f250bf-a1b5-4cec-b7fe-63862d33b94e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:22.888159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f250bf-a1b5-4cec-b7fe-63862d33b94e service swift] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:22.891888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda0ddf-3c74-4174-a114-c2cb06aa4e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:22.892426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda0ddf-3c74-4174-a114-c2cb06aa4e54 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:22.892707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda0ddf-3c74-4174-a114-c2cb06aa4e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:22.892981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda0ddf-3c74-4174-a114-c2cb06aa4e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:22.894743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-cfda0ddf-3c74-4174-a114-c2cb06aa4e54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:22.905052 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda0ddf-3c74-4174-a114-c2cb06aa4e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:22.906542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda0ddf-3c74-4174-a114-c2cb06aa4e54 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:22.914620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1181/2377] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:23.704807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=32 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:24.416933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9efc92-c81c-4ab7-ad0c-b6e6eb27c207 service nova] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:24.427110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9efc92-c81c-4ab7-ad0c-b6e6eb27c207 service nova] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:24.431442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce88efd-9042-4c90-8ccf-4931759ae0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:24.431442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce88efd-9042-4c90-8ccf-4931759ae0ba service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:24.431740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce88efd-9042-4c90-8ccf-4931759ae0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:24.431978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce88efd-9042-4c90-8ccf-4931759ae0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:24.434180 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3ce88efd-9042-4c90-8ccf-4931759ae0ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:24.444323 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce88efd-9042-4c90-8ccf-4931759ae0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:24.445716 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce88efd-9042-4c90-8ccf-4931759ae0ba service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:24.454404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1197/2378] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:25.061143 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda0ddf-3c74-4174-a114-c2cb06aa4e54 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:25.062757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-cfda0ddf-3c74-4174-a114-c2cb06aa4e54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:25.143624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda0ddf-3c74-4174-a114-c2cb06aa4e54 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:25.151378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f681454-80b5-4d26-80ab-5f01809a8b56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:25.151993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f681454-80b5-4d26-80ab-5f01809a8b56 service placement] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:25.152270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f681454-80b5-4d26-80ab-5f01809a8b56 service placement] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:25.152538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f681454-80b5-4d26-80ab-5f01809a8b56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:25.154295 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3f681454-80b5-4d26-80ab-5f01809a8b56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:25.166902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f681454-80b5-4d26-80ab-5f01809a8b56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:25.168554 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f681454-80b5-4d26-80ab-5f01809a8b56 service placement] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:25.178836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1182/2379] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:28.424929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce88efd-9042-4c90-8ccf-4931759ae0ba service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:28.441745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce88efd-9042-4c90-8ccf-4931759ae0ba service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:28.454503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628b933-4ce5-4129-a434-88d2bf58c99a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:28.455251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628b933-4ce5-4129-a434-88d2bf58c99a service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:28.455546 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628b933-4ce5-4129-a434-88d2bf58c99a service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:28.455826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628b933-4ce5-4129-a434-88d2bf58c99a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:28.457727 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a628b933-4ce5-4129-a434-88d2bf58c99a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:28.473635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628b933-4ce5-4129-a434-88d2bf58c99a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:28.475503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628b933-4ce5-4129-a434-88d2bf58c99a service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:28.489037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1198/2380] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:35.174465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=58 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:38.484343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=12 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:40.648427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f681454-80b5-4d26-80ab-5f01809a8b56 service placement] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:40.662721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f681454-80b5-4d26-80ab-5f01809a8b56 service placement] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:40.669531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314510a4-2d0b-4ca3-88c0-a5b13df77bf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:40.670204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314510a4-2d0b-4ca3-88c0-a5b13df77bf0 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:40.670485 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314510a4-2d0b-4ca3-88c0-a5b13df77bf0 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:40.670796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314510a4-2d0b-4ca3-88c0-a5b13df77bf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:40.672518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-314510a4-2d0b-4ca3-88c0-a5b13df77bf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:40.685404 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314510a4-2d0b-4ca3-88c0-a5b13df77bf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:40.687119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314510a4-2d0b-4ca3-88c0-a5b13df77bf0 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:40.709814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1183/2381] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:44.376373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc601e-09d2-4066-875d-f48a7b0d6c2f None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:44.376833 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc601e-09d2-4066-875d-f48a7b0d6c2f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:44.376948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc601e-09d2-4066-875d-f48a7b0d6c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:44.395405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-fbfc601e-09d2-4066-875d-f48a7b0d6c2f None None] MFA Rules not processed for user `acf57eb971eb4f78b173a5ab47160bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:51:44.396583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fbfc601e-09d2-4066-875d-f48a7b0d6c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:44.403828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1199/2382] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:51:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:51:46.338620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314510a4-2d0b-4ca3-88c0-a5b13df77bf0 service nova] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:46.349119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314510a4-2d0b-4ca3-88c0-a5b13df77bf0 service nova] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:46.353515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c09cf-ec07-4178-8ca3-98533ab52df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:46.355004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c09cf-ec07-4178-8ca3-98533ab52df1 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:46.355929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c09cf-ec07-4178-8ca3-98533ab52df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:46.356214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c09cf-ec07-4178-8ca3-98533ab52df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:46.365944 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c09cf-ec07-4178-8ca3-98533ab52df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:46.367437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c09cf-ec07-4178-8ca3-98533ab52df1 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:46.377383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1184/2383] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:48.187696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc601e-09d2-4066-875d-f48a7b0d6c2f None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:51:48.200948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc601e-09d2-4066-875d-f48a7b0d6c2f None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:51:48.206671 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad13fd6-d5fb-418e-b6b0-c421d1b2463f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:51:48.207270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad13fd6-d5fb-418e-b6b0-c421d1b2463f service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:51:48.207548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad13fd6-d5fb-418e-b6b0-c421d1b2463f service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:51:48.207821 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad13fd6-d5fb-418e-b6b0-c421d1b2463f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:51:48.209545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2ad13fd6-d5fb-418e-b6b0-c421d1b2463f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:51:48.221948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad13fd6-d5fb-418e-b6b0-c421d1b2463f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:51:48.223518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad13fd6-d5fb-418e-b6b0-c421d1b2463f service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:51:48.235385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1200/2384] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:51:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:51:56.373467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=12 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:51:58.229948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=9 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:01.525624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c09cf-ec07-4178-8ca3-98533ab52df1 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:01.537832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c09cf-ec07-4178-8ca3-98533ab52df1 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:01.543859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af22a44-b3bf-40c4-bf0f-60718e9b7c74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:01.544476 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af22a44-b3bf-40c4-bf0f-60718e9b7c74 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:01.544777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af22a44-b3bf-40c4-bf0f-60718e9b7c74 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:01.545038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af22a44-b3bf-40c4-bf0f-60718e9b7c74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:01.546948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8af22a44-b3bf-40c4-bf0f-60718e9b7c74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:01.606439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af22a44-b3bf-40c4-bf0f-60718e9b7c74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:01.608034 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af22a44-b3bf-40c4-bf0f-60718e9b7c74 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:01.616988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1185/2385] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:11.612747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=28 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:16.269905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35d653-a19c-4c0c-876b-0b8d2274c116 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:16.270465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35d653-a19c-4c0c-876b-0b8d2274c116 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:16.270575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba35d653-a19c-4c0c-876b-0b8d2274c116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:16.290431 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-ba35d653-a19c-4c0c-876b-0b8d2274c116 None None] MFA Rules not processed for user `3c0b7528b0ff4b09923a0e951d052ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:16.292122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ba35d653-a19c-4c0c-876b-0b8d2274c116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:16.299126 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1201/2386] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:52:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:16.602229 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af22a44-b3bf-40c4-bf0f-60718e9b7c74 service nova] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:52:16.612077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af22a44-b3bf-40c4-bf0f-60718e9b7c74 service nova] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:52:16.615685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3684dc8-5723-4710-a711-a055a9e50841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:52:16.616273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3684dc8-5723-4710-a711-a055a9e50841 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:16.616548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3684dc8-5723-4710-a711-a055a9e50841 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:16.616839 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3684dc8-5723-4710-a711-a055a9e50841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:16.618548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a3684dc8-5723-4710-a711-a055a9e50841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:16.627678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3684dc8-5723-4710-a711-a055a9e50841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:52:16.629081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3684dc8-5723-4710-a711-a055a9e50841 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:52:16.636711 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1186/2387] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:52:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:52:26.285569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:26.633307 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=6 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:52:37.313670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26447ef-ab7c-411a-ba9c-af798b300315 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:52:37.314193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26447ef-ab7c-411a-ba9c-af798b300315 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:52:37.314294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26447ef-ab7c-411a-ba9c-af798b300315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:52:37.333598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-a26447ef-ab7c-411a-ba9c-af798b300315 None None] MFA Rules not processed for user `ed4b7bfb87b144b38b82514c17a468bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:52:37.334839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a26447ef-ab7c-411a-ba9c-af798b300315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:52:37.342226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1202/2388] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:52:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:52:47.328446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:05.853247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3684dc8-5723-4710-a711-a055a9e50841 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:05.863535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3684dc8-5723-4710-a711-a055a9e50841 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:05.867851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904e998-7eeb-47a4-8602-1c759b1307e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:05.868665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904e998-7eeb-47a4-8602-1c759b1307e5 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:05.868933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904e998-7eeb-47a4-8602-1c759b1307e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:05.870925 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904e998-7eeb-47a4-8602-1c759b1307e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:05.874897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2904e998-7eeb-47a4-8602-1c759b1307e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:05.947184 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904e998-7eeb-47a4-8602-1c759b1307e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:05.948657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904e998-7eeb-47a4-8602-1c759b1307e5 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:05.958841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1187/2389] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:53:15.953859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=28 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:20.767958 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03296c3a-8e49-4d30-b166-ea2fbfcce026 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:20.768452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03296c3a-8e49-4d30-b166-ea2fbfcce026 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:20.768542 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03296c3a-8e49-4d30-b166-ea2fbfcce026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:20.794472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-03296c3a-8e49-4d30-b166-ea2fbfcce026 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:20.796237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-03296c3a-8e49-4d30-b166-ea2fbfcce026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:20.807588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1203/2390] 10.222.0.26 () {58 vars in 967 bytes} [Tue Oct 4 08:53:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:20.938310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904e998-7eeb-47a4-8602-1c759b1307e5 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:20.952709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904e998-7eeb-47a4-8602-1c759b1307e5 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:20.960707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27c30f-5bb8-411f-86bc-95bbf25ebb06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:20.961329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27c30f-5bb8-411f-86bc-95bbf25ebb06 service swift] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:20.962127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27c30f-5bb8-411f-86bc-95bbf25ebb06 service swift] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:20.962415 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27c30f-5bb8-411f-86bc-95bbf25ebb06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:20.972303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27c30f-5bb8-411f-86bc-95bbf25ebb06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:20.974002 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27c30f-5bb8-411f-86bc-95bbf25ebb06 service swift] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:20.983321 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1188/2391] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:53:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:53:21.431947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03296c3a-8e49-4d30-b166-ea2fbfcce026 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:21.452336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03296c3a-8e49-4d30-b166-ea2fbfcce026 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:21.456550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c1eed-1b3f-4a59-b06c-7d0411ca897e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:21.457463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c1eed-1b3f-4a59-b06c-7d0411ca897e service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:21.458101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c1eed-1b3f-4a59-b06c-7d0411ca897e service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:21.458586 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c1eed-1b3f-4a59-b06c-7d0411ca897e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:21.462136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2c7c1eed-1b3f-4a59-b06c-7d0411ca897e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:21.545666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c1eed-1b3f-4a59-b06c-7d0411ca897e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:21.547315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c1eed-1b3f-4a59-b06c-7d0411ca897e service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:21.558593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1204/2392] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:22.238186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d15bb-379f-4aa5-b489-8a106384822d None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:22.239003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d15bb-379f-4aa5-b489-8a106384822d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:22.239286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d15bb-379f-4aa5-b489-8a106384822d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:22.260228 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-b46d15bb-379f-4aa5-b489-8a106384822d None None] MFA Rules not processed for user `f928852d90b94ac5979e4a7806896b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:22.261436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b46d15bb-379f-4aa5-b489-8a106384822d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:22.316926 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1189/2393] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:53:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:25.233008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c1eed-1b3f-4a59-b06c-7d0411ca897e service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:25.243876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c1eed-1b3f-4a59-b06c-7d0411ca897e service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:25.248703 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb4563-59b0-4bfe-b65a-147dbfeb9c79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:25.249303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb4563-59b0-4bfe-b65a-147dbfeb9c79 service placement] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:25.249583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb4563-59b0-4bfe-b65a-147dbfeb9c79 service placement] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:25.249892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb4563-59b0-4bfe-b65a-147dbfeb9c79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:25.251577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-86bb4563-59b0-4bfe-b65a-147dbfeb9c79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:25.262968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb4563-59b0-4bfe-b65a-147dbfeb9c79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:25.264538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb4563-59b0-4bfe-b65a-147dbfeb9c79 service placement] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:25.275740 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1205/2394] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:28.881367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d15bb-379f-4aa5-b489-8a106384822d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:28.896706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d15bb-379f-4aa5-b489-8a106384822d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:28.903065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e005e-0dee-4f1c-9fd3-1f86fcfce512 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:28.903856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e005e-0dee-4f1c-9fd3-1f86fcfce512 service placement] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:28.904296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e005e-0dee-4f1c-9fd3-1f86fcfce512 service placement] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:28.904783 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e005e-0dee-4f1c-9fd3-1f86fcfce512 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:28.907178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-709e005e-0dee-4f1c-9fd3-1f86fcfce512 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:28.921878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e005e-0dee-4f1c-9fd3-1f86fcfce512 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:28.923522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e005e-0dee-4f1c-9fd3-1f86fcfce512 service placement] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:28.941555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb4563-59b0-4bfe-b65a-147dbfeb9c79 service placement] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:28.945890 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1190/2395] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:28.947139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-86bb4563-59b0-4bfe-b65a-147dbfeb9c79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:28.963407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb4563-59b0-4bfe-b65a-147dbfeb9c79 service placement] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:28.971811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:28.972650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:28.972928 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:28.973388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] PATH_INFO: `/v3/users/8fcf4142aa8340d692b1b3ed0aee7c87` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:28.974563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fcf4142aa8340d692b1b3ed0aee7c87)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:28.976571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:29.030172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fcf4142aa8340d692b1b3ed0aee7c87'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.030818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fcf4142aa8340d692b1b3ed0aee7c87'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.037735 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fcf4142aa8340d692b1b3ed0aee7c87'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.038358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fcf4142aa8340d692b1b3ed0aee7c87'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.050107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fcf4142aa8340d692b1b3ed0aee7c87'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.062866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1206/2396] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:28 2022] DELETE /identity/v3/users/8fcf4142aa8340d692b1b3ed0aee7c87 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:29.251087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e005e-0dee-4f1c-9fd3-1f86fcfce512 service placement] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:29.266105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e005e-0dee-4f1c-9fd3-1f86fcfce512 service placement] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:29.326376 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:29.327008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:29.327296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:29.327573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] PATH_INFO: `/v3/projects/fe42bae7e14246d78103a4ef69457051` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:29.328689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe42bae7e14246d78103a4ef69457051)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:29.330725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:29.379309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe42bae7e14246d78103a4ef69457051', 'request_id': 'req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.386873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe42bae7e14246d78103a4ef69457051', 'request_id': 'req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.392762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe42bae7e14246d78103a4ef69457051', 'request_id': 'req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.406324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] The token cache is being invalidate because project fe42bae7e14246d78103a4ef69457051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:29.406620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.407296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.408832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1191/2397] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:29 2022] DELETE /identity/v3/projects/fe42bae7e14246d78103a4ef69457051 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:29.413512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:29.415046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:29.429160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c98ec5-e3f7-46f4-9757-f56c27fa27e8 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:29.477536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:29.495728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:29.496371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:29.496650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:29.496948 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] PATH_INFO: `/v3/users/d33ac816c90d458788c5fa7ab2680bfb` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:29.498105 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d33ac816c90d458788c5fa7ab2680bfb)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:29.500081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:29.553374 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd33ac816c90d458788c5fa7ab2680bfb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.554071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd33ac816c90d458788c5fa7ab2680bfb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.561447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd33ac816c90d458788c5fa7ab2680bfb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.562071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd33ac816c90d458788c5fa7ab2680bfb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.575662 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd33ac816c90d458788c5fa7ab2680bfb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.588865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1207/2398] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:29 2022] DELETE /identity/v3/users/d33ac816c90d458788c5fa7ab2680bfb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:29.761477 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:29.763191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:29.780517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296f7c4-2675-4e3f-b3bf-7e8c71751ed3 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:29.786105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:29.843879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:29.844564 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:29.844857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:29.845139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] PATH_INFO: `/v3/projects/4c74255a5781460facaa444c24f2ef86` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:29.846410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c74255a5781460facaa444c24f2ef86)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:29.848524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:29.900749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c74255a5781460facaa444c24f2ef86', 'request_id': 'req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.907423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c74255a5781460facaa444c24f2ef86', 'request_id': 'req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.914348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c74255a5781460facaa444c24f2ef86', 'request_id': 'req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.927975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] The token cache is being invalidate because project 4c74255a5781460facaa444c24f2ef86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:29.927975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.928562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:29.930071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1192/2399] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:29 2022] DELETE /identity/v3/projects/4c74255a5781460facaa444c24f2ef86 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:32.394587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:32.396036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:32.455188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353eaa12-c370-4337-b53d-f5e29307e951 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:32.504672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1500b1-0528-4ae8-83b0-c6d5194e57ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:32.505234 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1500b1-0528-4ae8-83b0-c6d5194e57ef service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:32.505513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1500b1-0528-4ae8-83b0-c6d5194e57ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:32.505845 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1500b1-0528-4ae8-83b0-c6d5194e57ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:32.507589 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6c1500b1-0528-4ae8-83b0-c6d5194e57ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:32.544050 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:32.575607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1500b1-0528-4ae8-83b0-c6d5194e57ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:32.577081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1500b1-0528-4ae8-83b0-c6d5194e57ef service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:32.587144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1208/2400] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:35.862289 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:35.863866 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:35.880872 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a59fea-9d3c-45af-a1e3-01816bcf1d0f admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:35.885709 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65658387-6a2e-4d3a-a3ad-03a519c2063e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:35.886357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65658387-6a2e-4d3a-a3ad-03a519c2063e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:35.886647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65658387-6a2e-4d3a-a3ad-03a519c2063e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:35.886943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65658387-6a2e-4d3a-a3ad-03a519c2063e admin admin] PATH_INFO: `/v3/users/acf57eb971eb4f78b173a5ab47160bd1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:35.888104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65658387-6a2e-4d3a-a3ad-03a519c2063e admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf57eb971eb4f78b173a5ab47160bd1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:35.890083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65658387-6a2e-4d3a-a3ad-03a519c2063e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:35.936077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-65658387-6a2e-4d3a-a3ad-03a519c2063e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf57eb971eb4f78b173a5ab47160bd1'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:35.936727 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-65658387-6a2e-4d3a-a3ad-03a519c2063e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf57eb971eb4f78b173a5ab47160bd1'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:35.948897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-65658387-6a2e-4d3a-a3ad-03a519c2063e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf57eb971eb4f78b173a5ab47160bd1'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:35.955897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-65658387-6a2e-4d3a-a3ad-03a519c2063e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf57eb971eb4f78b173a5ab47160bd1'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:35.956555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-65658387-6a2e-4d3a-a3ad-03a519c2063e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf57eb971eb4f78b173a5ab47160bd1'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:35.969610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1193/2401] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:35 2022] DELETE /identity/v3/users/acf57eb971eb4f78b173a5ab47160bd1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:36.152671 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1500b1-0528-4ae8-83b0-c6d5194e57ef service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.163271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1500b1-0528-4ae8-83b0-c6d5194e57ef service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.226452 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730be9fc-766a-4289-a4c2-90b74196d736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.227085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730be9fc-766a-4289-a4c2-90b74196d736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.227481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730be9fc-766a-4289-a4c2-90b74196d736 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.227761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730be9fc-766a-4289-a4c2-90b74196d736 admin admin] PATH_INFO: `/v3/projects/0b7703aae5634dabb496bc221882863c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.228970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730be9fc-766a-4289-a4c2-90b74196d736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b7703aae5634dabb496bc221882863c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.230977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730be9fc-766a-4289-a4c2-90b74196d736 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.404261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-730be9fc-766a-4289-a4c2-90b74196d736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b7703aae5634dabb496bc221882863c', 'request_id': 'req-730be9fc-766a-4289-a4c2-90b74196d736'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.410932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-730be9fc-766a-4289-a4c2-90b74196d736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b7703aae5634dabb496bc221882863c', 'request_id': 'req-730be9fc-766a-4289-a4c2-90b74196d736'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.417314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-730be9fc-766a-4289-a4c2-90b74196d736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b7703aae5634dabb496bc221882863c', 'request_id': 'req-730be9fc-766a-4289-a4c2-90b74196d736'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.430424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-730be9fc-766a-4289-a4c2-90b74196d736 admin admin] The token cache is being invalidate because project 0b7703aae5634dabb496bc221882863c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:36.430714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-730be9fc-766a-4289-a4c2-90b74196d736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.431220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-730be9fc-766a-4289-a4c2-90b74196d736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:36.433673 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1209/2402] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:36 2022] DELETE /identity/v3/projects/0b7703aae5634dabb496bc221882863c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:36.439788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4da623-d814-430b-9f02-2dc83dcd7b50 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.440075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4da623-d814-430b-9f02-2dc83dcd7b50 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.440505 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4da623-d814-430b-9f02-2dc83dcd7b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.461254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-2d4da623-d814-430b-9f02-2dc83dcd7b50 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:36.462470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2d4da623-d814-430b-9f02-2dc83dcd7b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:36.467654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:36.522346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1194/2403] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:36.530479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f360e-72a1-4686-8f0b-5c95b627460c None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.530770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f360e-72a1-4686-8f0b-5c95b627460c None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.531046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f360e-72a1-4686-8f0b-5c95b627460c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.550030 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-609f360e-72a1-4686-8f0b-5c95b627460c None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:36.551228 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-609f360e-72a1-4686-8f0b-5c95b627460c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:36.559515 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1210/2404] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:36.563607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4da623-d814-430b-9f02-2dc83dcd7b50 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.573865 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4da623-d814-430b-9f02-2dc83dcd7b50 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.578503 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a81fb-2abf-4e72-b077-c4f7e4fb8df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.579123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a81fb-2abf-4e72-b077-c4f7e4fb8df6 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.579402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a81fb-2abf-4e72-b077-c4f7e4fb8df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.579673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a81fb-2abf-4e72-b077-c4f7e4fb8df6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.580309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a81fb-2abf-4e72-b077-c4f7e4fb8df6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:53:36.580637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a81fb-2abf-4e72-b077-c4f7e4fb8df6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.582273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a81fb-2abf-4e72-b077-c4f7e4fb8df6 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.593713 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1195/2405] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:53:36.597851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f360e-72a1-4686-8f0b-5c95b627460c None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.607695 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f360e-72a1-4686-8f0b-5c95b627460c None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.612677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a9e53-a487-4cac-a2e9-9793dff7331f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.613310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a9e53-a487-4cac-a2e9-9793dff7331f admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.613615 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a9e53-a487-4cac-a2e9-9793dff7331f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.613920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a9e53-a487-4cac-a2e9-9793dff7331f admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.614602 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a9e53-a487-4cac-a2e9-9793dff7331f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.617210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a9e53-a487-4cac-a2e9-9793dff7331f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.644544 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1211/2406] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:36 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:36.648771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a81fb-2abf-4e72-b077-c4f7e4fb8df6 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.659356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a81fb-2abf-4e72-b077-c4f7e4fb8df6 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.716244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1588be-73bd-4b4a-9744-49cb49089804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.716894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1588be-73bd-4b4a-9744-49cb49089804 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.717200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1588be-73bd-4b4a-9744-49cb49089804 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.717475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1588be-73bd-4b4a-9744-49cb49089804 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.718135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1588be-73bd-4b4a-9744-49cb49089804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.720100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1588be-73bd-4b4a-9744-49cb49089804 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.740970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1196/2407] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:36 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:36.745081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a9e53-a487-4cac-a2e9-9793dff7331f admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.756631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a9e53-a487-4cac-a2e9-9793dff7331f admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.761441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740ec7a-58cc-4ab0-8270-6d23a87d3075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.762111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740ec7a-58cc-4ab0-8270-6d23a87d3075 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.762447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740ec7a-58cc-4ab0-8270-6d23a87d3075 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.762765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740ec7a-58cc-4ab0-8270-6d23a87d3075 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.763459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740ec7a-58cc-4ab0-8270-6d23a87d3075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.765329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740ec7a-58cc-4ab0-8270-6d23a87d3075 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.776153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1212/2408] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:36.784937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1588be-73bd-4b4a-9744-49cb49089804 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.795786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1588be-73bd-4b4a-9744-49cb49089804 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.800471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcdce6-0602-4d2b-8c22-0d8799e72423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.801067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcdce6-0602-4d2b-8c22-0d8799e72423 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.801352 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcdce6-0602-4d2b-8c22-0d8799e72423 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.801625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcdce6-0602-4d2b-8c22-0d8799e72423 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.802297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcdce6-0602-4d2b-8c22-0d8799e72423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.803901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcdce6-0602-4d2b-8c22-0d8799e72423 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.815285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1197/2409] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:36.819633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740ec7a-58cc-4ab0-8270-6d23a87d3075 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.829576 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740ec7a-58cc-4ab0-8270-6d23a87d3075 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:36.834812 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783561e3-8ab4-4657-b049-0f7c3467806f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:36.835718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783561e3-8ab4-4657-b049-0f7c3467806f admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.835997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783561e3-8ab4-4657-b049-0f7c3467806f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.836271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783561e3-8ab4-4657-b049-0f7c3467806f admin admin] PATH_INFO: `/v3/projects/3aba5810875541cea0a06fd5777062ff/users/9f3a1a3d6fd745418f23a2a50a704dd3/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.851423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783561e3-8ab4-4657-b049-0f7c3467806f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aba5810875541cea0a06fd5777062ff, user_id=9f3a1a3d6fd745418f23a2a50a704dd3, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:36.854139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783561e3-8ab4-4657-b049-0f7c3467806f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:36.862192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:36.869503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1213/2410] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:36 2022] PUT /identity/v3/projects/3aba5810875541cea0a06fd5777062ff/users/9f3a1a3d6fd745418f23a2a50a704dd3/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:36.881063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ebab0-c9a3-46bf-bda0-3e93802936fb None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:36.881542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ebab0-c9a3-46bf-bda0-3e93802936fb None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:36.881640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ebab0-c9a3-46bf-bda0-3e93802936fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:36.909437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-811ebab0-c9a3-46bf-bda0-3e93802936fb None None] MFA Rules not processed for user `9f3a1a3d6fd745418f23a2a50a704dd3`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:36.910775 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-811ebab0-c9a3-46bf-bda0-3e93802936fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:36.973354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1198/2411] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:36.984536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783561e3-8ab4-4657-b049-0f7c3467806f admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:36.986308 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-783561e3-8ab4-4657-b049-0f7c3467806f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:37.058479 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783561e3-8ab4-4657-b049-0f7c3467806f admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:37.063810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024ede8-1bcc-48e7-8f22-56d562250ecd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:37.064402 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024ede8-1bcc-48e7-8f22-56d562250ecd service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:37.064677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024ede8-1bcc-48e7-8f22-56d562250ecd service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:37.064941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024ede8-1bcc-48e7-8f22-56d562250ecd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:37.073028 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024ede8-1bcc-48e7-8f22-56d562250ecd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:37.074638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0024ede8-1bcc-48e7-8f22-56d562250ecd service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:37.083336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1214/2412] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:39.819183 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ebab0-c9a3-46bf-bda0-3e93802936fb None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:39.826874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:39.883039 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ebab0-c9a3-46bf-bda0-3e93802936fb None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:39.887677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:39.888451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:39.888734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:39.889009 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] PATH_INFO: `/v3/users/9f3a1a3d6fd745418f23a2a50a704dd3` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:39.890231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f3a1a3d6fd745418f23a2a50a704dd3)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:39.892459 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:39.942552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f3a1a3d6fd745418f23a2a50a704dd3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:39.943254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f3a1a3d6fd745418f23a2a50a704dd3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:39.957846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f3a1a3d6fd745418f23a2a50a704dd3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:39.967967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f3a1a3d6fd745418f23a2a50a704dd3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:39.968690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f3a1a3d6fd745418f23a2a50a704dd3'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:39.982720 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1199/2413] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:53:39 2022] DELETE /identity/v3/users/9f3a1a3d6fd745418f23a2a50a704dd3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:39.987320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8c3eb-d425-4810-bba6-971ec3eacf88 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:39.987803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8c3eb-d425-4810-bba6-971ec3eacf88 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:39.987910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8c3eb-d425-4810-bba6-971ec3eacf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.007746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-f1c8c3eb-d425-4810-bba6-971ec3eacf88 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:40.008942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f1c8c3eb-d425-4810-bba6-971ec3eacf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:40.065632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1215/2414] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:53:40.075362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.076799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:40.139241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09beb310-04ff-4b0f-99e7-2a639d8eb0bd admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.143024 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63c38c-2ec9-477d-9dd5-3404374b4ff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.143622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63c38c-2ec9-477d-9dd5-3404374b4ff1 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.143899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63c38c-2ec9-477d-9dd5-3404374b4ff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.144170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63c38c-2ec9-477d-9dd5-3404374b4ff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.153180 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63c38c-2ec9-477d-9dd5-3404374b4ff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.154606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63c38c-2ec9-477d-9dd5-3404374b4ff1 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.164347 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1200/2415] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:40.432830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8c3eb-d425-4810-bba6-971ec3eacf88 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.434331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f1c8c3eb-d425-4810-bba6-971ec3eacf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:40.514814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8c3eb-d425-4810-bba6-971ec3eacf88 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.519423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.520045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.520321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.520589 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e admin admin] PATH_INFO: `/v3/projects/3aba5810875541cea0a06fd5777062ff` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.521680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aba5810875541cea0a06fd5777062ff)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.523588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.539423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:40.569876 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aba5810875541cea0a06fd5777062ff', 'request_id': 'req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:40.576197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aba5810875541cea0a06fd5777062ff', 'request_id': 'req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:40.581823 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aba5810875541cea0a06fd5777062ff', 'request_id': 'req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:40.593987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e admin admin] The token cache is being invalidate because project 3aba5810875541cea0a06fd5777062ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:53:40.594322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:40.595047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-67e91a0c-5ef0-4d32-964b-8bc8dd75712e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:53:40.596493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1216/2416] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:53:40 2022] DELETE /identity/v3/projects/3aba5810875541cea0a06fd5777062ff => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:40.602028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da741f29-d49a-44a8-86d2-20bfbdd151b2 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.602316 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da741f29-d49a-44a8-86d2-20bfbdd151b2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.602591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da741f29-d49a-44a8-86d2-20bfbdd151b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.640246 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:40.660430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-da741f29-d49a-44a8-86d2-20bfbdd151b2 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:40.662147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-da741f29-d49a-44a8-86d2-20bfbdd151b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:40.720426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1201/2417] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:40.728737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87025d7b-17f1-4020-b809-7688ec9bdddd None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.729025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87025d7b-17f1-4020-b809-7688ec9bdddd None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.729305 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87025d7b-17f1-4020-b809-7688ec9bdddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.748806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-87025d7b-17f1-4020-b809-7688ec9bdddd None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:40.750012 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-87025d7b-17f1-4020-b809-7688ec9bdddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:40.757505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1217/2418] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:40.761670 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da741f29-d49a-44a8-86d2-20bfbdd151b2 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.771141 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da741f29-d49a-44a8-86d2-20bfbdd151b2 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.775618 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bccfa1-a6b0-42f2-b870-e576b1bedfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.776205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bccfa1-a6b0-42f2-b870-e576b1bedfa1 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.776484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bccfa1-a6b0-42f2-b870-e576b1bedfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.776757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bccfa1-a6b0-42f2-b870-e576b1bedfa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.777387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bccfa1-a6b0-42f2-b870-e576b1bedfa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:53:40.777704 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bccfa1-a6b0-42f2-b870-e576b1bedfa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.779296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bccfa1-a6b0-42f2-b870-e576b1bedfa1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.790895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1202/2419] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:53:40.795121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87025d7b-17f1-4020-b809-7688ec9bdddd None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.804708 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87025d7b-17f1-4020-b809-7688ec9bdddd None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.809335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa4778-781b-4b81-b2d4-e1ccf45e35a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.809947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa4778-781b-4b81-b2d4-e1ccf45e35a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.810230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa4778-781b-4b81-b2d4-e1ccf45e35a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.810504 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa4778-781b-4b81-b2d4-e1ccf45e35a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.811123 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa4778-781b-4b81-b2d4-e1ccf45e35a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.813016 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa4778-781b-4b81-b2d4-e1ccf45e35a9 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.840160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1218/2420] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:40 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:40.844287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bccfa1-a6b0-42f2-b870-e576b1bedfa1 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.854859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bccfa1-a6b0-42f2-b870-e576b1bedfa1 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.913786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48959eba-d13c-4e4d-aa17-001acef9ad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.914478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48959eba-d13c-4e4d-aa17-001acef9ad14 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.914765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48959eba-d13c-4e4d-aa17-001acef9ad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.915071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48959eba-d13c-4e4d-aa17-001acef9ad14 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.915708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48959eba-d13c-4e4d-aa17-001acef9ad14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.917707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48959eba-d13c-4e4d-aa17-001acef9ad14 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.941646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1203/2421] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:40 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:40.946817 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa4778-781b-4b81-b2d4-e1ccf45e35a9 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.956927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa4778-781b-4b81-b2d4-e1ccf45e35a9 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.962840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faf967-c11c-4078-9d93-6b391bc5ea20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.963520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faf967-c11c-4078-9d93-6b391bc5ea20 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.963798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faf967-c11c-4078-9d93-6b391bc5ea20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.964123 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faf967-c11c-4078-9d93-6b391bc5ea20 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.964770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57faf967-c11c-4078-9d93-6b391bc5ea20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:40.966494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57faf967-c11c-4078-9d93-6b391bc5ea20 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:40.976604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1219/2422] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:40.980787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48959eba-d13c-4e4d-aa17-001acef9ad14 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:40.991665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48959eba-d13c-4e4d-aa17-001acef9ad14 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:40.996606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d8f15c-bac9-4f9b-a7e2-d751f831226b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:40.997207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8f15c-bac9-4f9b-a7e2-d751f831226b admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:40.997485 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8f15c-bac9-4f9b-a7e2-d751f831226b admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:40.997812 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d8f15c-bac9-4f9b-a7e2-d751f831226b admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:40.998454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8f15c-bac9-4f9b-a7e2-d751f831226b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:41.000053 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d8f15c-bac9-4f9b-a7e2-d751f831226b admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:41.011426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1204/2423] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:41.015874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faf967-c11c-4078-9d93-6b391bc5ea20 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:41.026483 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faf967-c11c-4078-9d93-6b391bc5ea20 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:41.031203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a7388-c361-4990-bc7f-d2ca77451d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:41.031851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a7388-c361-4990-bc7f-d2ca77451d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.032169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a7388-c361-4990-bc7f-d2ca77451d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.032447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a7388-c361-4990-bc7f-d2ca77451d06 admin admin] PATH_INFO: `/v3/projects/587acaf3719f4f5ab8a3c2b4f970479c/users/893fb1ee9f954487859f53f14bd98a44/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.046718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a7388-c361-4990-bc7f-d2ca77451d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587acaf3719f4f5ab8a3c2b4f970479c, user_id=893fb1ee9f954487859f53f14bd98a44, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:41.049333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a7388-c361-4990-bc7f-d2ca77451d06 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:41.062045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1220/2424] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:41 2022] PUT /identity/v3/projects/587acaf3719f4f5ab8a3c2b4f970479c/users/893fb1ee9f954487859f53f14bd98a44/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:41.073079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c099a4-1d25-4137-ad39-06fb70e03f00 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.073398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c099a4-1d25-4137-ad39-06fb70e03f00 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.073658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c099a4-1d25-4137-ad39-06fb70e03f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.100792 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-08c099a4-1d25-4137-ad39-06fb70e03f00 None None] MFA Rules not processed for user `893fb1ee9f954487859f53f14bd98a44`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:41.102014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-08c099a4-1d25-4137-ad39-06fb70e03f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:41.155990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:41.167579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1205/2425] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:53:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:41.172115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a7388-c361-4990-bc7f-d2ca77451d06 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:41.231569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a7388-c361-4990-bc7f-d2ca77451d06 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:41.236210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14410e01-b979-4c7b-a31f-0b8d03a9d840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:41.236809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14410e01-b979-4c7b-a31f-0b8d03a9d840 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.237089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14410e01-b979-4c7b-a31f-0b8d03a9d840 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.237442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14410e01-b979-4c7b-a31f-0b8d03a9d840 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.238098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14410e01-b979-4c7b-a31f-0b8d03a9d840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:41.240712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14410e01-b979-4c7b-a31f-0b8d03a9d840 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:41.267209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1221/2426] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:53:41 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:41.271281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c099a4-1d25-4137-ad39-06fb70e03f00 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:41.282178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c099a4-1d25-4137-ad39-06fb70e03f00 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:41.337258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba1b6f-7ec7-4583-ac1c-052010f669b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:41.337884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba1b6f-7ec7-4583-ac1c-052010f669b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.338163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba1b6f-7ec7-4583-ac1c-052010f669b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.338432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba1b6f-7ec7-4583-ac1c-052010f669b7 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.339057 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba1b6f-7ec7-4583-ac1c-052010f669b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:41.341021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebba1b6f-7ec7-4583-ac1c-052010f669b7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:41.368495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1206/2427] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:53:41 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:53:41.370009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14410e01-b979-4c7b-a31f-0b8d03a9d840 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:41.381357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14410e01-b979-4c7b-a31f-0b8d03a9d840 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:41.386235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ac61b-e2ca-4eff-a4ce-73d725662a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:41.386833 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ac61b-e2ca-4eff-a4ce-73d725662a03 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.387112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ac61b-e2ca-4eff-a4ce-73d725662a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.387385 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ac61b-e2ca-4eff-a4ce-73d725662a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.388024 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ac61b-e2ca-4eff-a4ce-73d725662a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:41.389616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ac61b-e2ca-4eff-a4ce-73d725662a03 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:41.400691 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1222/2428] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:53:41.404843 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba1b6f-7ec7-4583-ac1c-052010f669b7 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:41.414634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba1b6f-7ec7-4583-ac1c-052010f669b7 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:41.419267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60276825-93e5-43a5-a05c-1fbebdb45981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:41.419847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60276825-93e5-43a5-a05c-1fbebdb45981 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.420123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60276825-93e5-43a5-a05c-1fbebdb45981 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.420390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60276825-93e5-43a5-a05c-1fbebdb45981 admin admin] PATH_INFO: `/v3/projects/6953785191ba4f5a99abad5c6ed914f1/users/6a9cad2dda4942b0bbcd20ffc8725230/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.434776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60276825-93e5-43a5-a05c-1fbebdb45981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6953785191ba4f5a99abad5c6ed914f1, user_id=6a9cad2dda4942b0bbcd20ffc8725230, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:41.437417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60276825-93e5-43a5-a05c-1fbebdb45981 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:41.450259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1207/2429] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:41 2022] PUT /identity/v3/projects/6953785191ba4f5a99abad5c6ed914f1/users/6a9cad2dda4942b0bbcd20ffc8725230/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:41.454079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ac61b-e2ca-4eff-a4ce-73d725662a03 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:41.465514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ac61b-e2ca-4eff-a4ce-73d725662a03 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:41.476841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:41.527228 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabd9c2-4f6f-470b-8537-d90c6aae0e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:41.527840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabd9c2-4f6f-470b-8537-d90c6aae0e0f admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.528171 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabd9c2-4f6f-470b-8537-d90c6aae0e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.528488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabd9c2-4f6f-470b-8537-d90c6aae0e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.529181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabd9c2-4f6f-470b-8537-d90c6aae0e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:41.531007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabd9c2-4f6f-470b-8537-d90c6aae0e0f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:41.541935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1223/2430] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:53:41.546335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60276825-93e5-43a5-a05c-1fbebdb45981 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:41.570160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60276825-93e5-43a5-a05c-1fbebdb45981 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:41.577273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fde8a2-a7f3-4dee-9f03-fc631fbc8049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:41.578228 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fde8a2-a7f3-4dee-9f03-fc631fbc8049 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.578565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fde8a2-a7f3-4dee-9f03-fc631fbc8049 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.580633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fde8a2-a7f3-4dee-9f03-fc631fbc8049 admin admin] PATH_INFO: `/v3/projects/6953785191ba4f5a99abad5c6ed914f1/users/6a9cad2dda4942b0bbcd20ffc8725230/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.580856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fde8a2-a7f3-4dee-9f03-fc631fbc8049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6953785191ba4f5a99abad5c6ed914f1, user_id=6a9cad2dda4942b0bbcd20ffc8725230, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:41.584120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fde8a2-a7f3-4dee-9f03-fc631fbc8049 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:41.595021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1208/2431] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:53:41 2022] PUT /identity/v3/projects/6953785191ba4f5a99abad5c6ed914f1/users/6a9cad2dda4942b0bbcd20ffc8725230/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:53:41.606888 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce978f-7b34-4f4a-8bb3-a0c453426e3f None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.607182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce978f-7b34-4f4a-8bb3-a0c453426e3f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.607458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce978f-7b34-4f4a-8bb3-a0c453426e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.635369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-a5ce978f-7b34-4f4a-8bb3-a0c453426e3f None None] MFA Rules not processed for user `6a9cad2dda4942b0bbcd20ffc8725230`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:53:41.636580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a5ce978f-7b34-4f4a-8bb3-a0c453426e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:41.702418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1224/2432] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:53:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:53:41.713901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fde8a2-a7f3-4dee-9f03-fc631fbc8049 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:41.715590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-73fde8a2-a7f3-4dee-9f03-fc631fbc8049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:53:41.788799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fde8a2-a7f3-4dee-9f03-fc631fbc8049 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:41.792475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def0df5-06b0-4641-a25c-600220d26860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:41.793073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def0df5-06b0-4641-a25c-600220d26860 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.793358 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def0df5-06b0-4641-a25c-600220d26860 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.793634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def0df5-06b0-4641-a25c-600220d26860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.802480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def0df5-06b0-4641-a25c-600220d26860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:41.803903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def0df5-06b0-4641-a25c-600220d26860 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:41.813279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1209/2433] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:41.889116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce978f-7b34-4f4a-8bb3-a0c453426e3f None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:53:41.899268 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce978f-7b34-4f4a-8bb3-a0c453426e3f None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:53:41.902803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9077993-bb05-4b0f-a5fc-7d79e9632d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:53:41.903397 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9077993-bb05-4b0f-a5fc-7d79e9632d59 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:53:41.903693 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9077993-bb05-4b0f-a5fc-7d79e9632d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:53:41.903944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9077993-bb05-4b0f-a5fc-7d79e9632d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:53:41.959968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9077993-bb05-4b0f-a5fc-7d79e9632d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:53:41.961307 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9077993-bb05-4b0f-a5fc-7d79e9632d59 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:53:41.970021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1225/2434] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:53:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:53:51.809035 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=80,keystone:INSERT=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:53:51.965970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=84 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:27.524134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def0df5-06b0-4641-a25c-600220d26860 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:27.594783 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def0df5-06b0-4641-a25c-600220d26860 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:27.599447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:27.600131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:27.600555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:27.600836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] PATH_INFO: `/v3/users/893fb1ee9f954487859f53f14bd98a44` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:27.602041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=893fb1ee9f954487859f53f14bd98a44)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:27.604019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:27.656552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893fb1ee9f954487859f53f14bd98a44'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.656552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893fb1ee9f954487859f53f14bd98a44'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.675494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893fb1ee9f954487859f53f14bd98a44'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.682102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893fb1ee9f954487859f53f14bd98a44'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.682858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893fb1ee9f954487859f53f14bd98a44'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:27.696843 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1210/2435] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:27 2022] DELETE /identity/v3/users/893fb1ee9f954487859f53f14bd98a44 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:27.701064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc0af7-8e8c-4b91-b38f-eef4c057f2df None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:27.701630 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc0af7-8e8c-4b91-b38f-eef4c057f2df None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:27.701779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc0af7-8e8c-4b91-b38f-eef4c057f2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:27.720742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-85dc0af7-8e8c-4b91-b38f-eef4c057f2df None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:27.722005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-85dc0af7-8e8c-4b91-b38f-eef4c057f2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:27.785271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1226/2436] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:27.794780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:27.796176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:27.877294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd40bb1-922c-49d7-ba17-bbd023af91d2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:27.881056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b85c7-82ea-448e-b329-56d0c33d4524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:27.881662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b85c7-82ea-448e-b329-56d0c33d4524 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:27.881970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b85c7-82ea-448e-b329-56d0c33d4524 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:27.882243 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b85c7-82ea-448e-b329-56d0c33d4524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:27.891035 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b85c7-82ea-448e-b329-56d0c33d4524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:27.892469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b85c7-82ea-448e-b329-56d0c33d4524 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:27.901745 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1211/2437] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:28.165054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc0af7-8e8c-4b91-b38f-eef4c057f2df None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:28.166521 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-85dc0af7-8e8c-4b91-b38f-eef4c057f2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:28.181224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc0af7-8e8c-4b91-b38f-eef4c057f2df None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:28.185897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:28.186530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:28.186799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:28.187080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] PATH_INFO: `/v3/projects/587acaf3719f4f5ab8a3c2b4f970479c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:28.188197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=587acaf3719f4f5ab8a3c2b4f970479c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:28.190145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:28.238491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587acaf3719f4f5ab8a3c2b4f970479c', 'request_id': 'req-8b20ec83-d114-4ae4-a461-090126ac5119'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.245149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587acaf3719f4f5ab8a3c2b4f970479c', 'request_id': 'req-8b20ec83-d114-4ae4-a461-090126ac5119'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.251302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587acaf3719f4f5ab8a3c2b4f970479c', 'request_id': 'req-8b20ec83-d114-4ae4-a461-090126ac5119'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.264227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] The token cache is being invalidate because project 587acaf3719f4f5ab8a3c2b4f970479c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:28.264518 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.265183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.266780 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1227/2438] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:28 2022] DELETE /identity/v3/projects/587acaf3719f4f5ab8a3c2b4f970479c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:28.271315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b85c7-82ea-448e-b329-56d0c33d4524 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:28.276270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:28.282606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b85c7-82ea-448e-b329-56d0c33d4524 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:28.342852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7fee8-4137-4784-b868-7a4135f91a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:28.343543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7fee8-4137-4784-b868-7a4135f91a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:28.343825 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7fee8-4137-4784-b868-7a4135f91a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:28.344099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7fee8-4137-4784-b868-7a4135f91a23 admin admin] PATH_INFO: `/v3/users/6a9cad2dda4942b0bbcd20ffc8725230` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:28.345250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7fee8-4137-4784-b868-7a4135f91a23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9cad2dda4942b0bbcd20ffc8725230)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:28.347857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7fee8-4137-4784-b868-7a4135f91a23 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:28.395107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3ca7fee8-4137-4784-b868-7a4135f91a23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9cad2dda4942b0bbcd20ffc8725230'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.395746 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3ca7fee8-4137-4784-b868-7a4135f91a23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9cad2dda4942b0bbcd20ffc8725230'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.407269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3ca7fee8-4137-4784-b868-7a4135f91a23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9cad2dda4942b0bbcd20ffc8725230'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.414525 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3ca7fee8-4137-4784-b868-7a4135f91a23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9cad2dda4942b0bbcd20ffc8725230'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.415153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3ca7fee8-4137-4784-b868-7a4135f91a23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9cad2dda4942b0bbcd20ffc8725230'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.427188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1212/2439] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:28 2022] DELETE /identity/v3/users/6a9cad2dda4942b0bbcd20ffc8725230 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:28.687223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:28.688643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:28.703562 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20ec83-d114-4ae4-a461-090126ac5119 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:28.760532 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cee44a-7dfa-4164-a33c-57802a443032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:28.761152 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cee44a-7dfa-4164-a33c-57802a443032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:28.761428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cee44a-7dfa-4164-a33c-57802a443032 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:28.761707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cee44a-7dfa-4164-a33c-57802a443032 admin admin] PATH_INFO: `/v3/projects/6953785191ba4f5a99abad5c6ed914f1` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:28.762819 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cee44a-7dfa-4164-a33c-57802a443032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6953785191ba4f5a99abad5c6ed914f1)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:28.764705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cee44a-7dfa-4164-a33c-57802a443032 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:28.793841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:28.812064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a3cee44a-7dfa-4164-a33c-57802a443032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6953785191ba4f5a99abad5c6ed914f1', 'request_id': 'req-a3cee44a-7dfa-4164-a33c-57802a443032'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.818877 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a3cee44a-7dfa-4164-a33c-57802a443032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6953785191ba4f5a99abad5c6ed914f1', 'request_id': 'req-a3cee44a-7dfa-4164-a33c-57802a443032'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.824482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a3cee44a-7dfa-4164-a33c-57802a443032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6953785191ba4f5a99abad5c6ed914f1', 'request_id': 'req-a3cee44a-7dfa-4164-a33c-57802a443032'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.838045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a3cee44a-7dfa-4164-a33c-57802a443032 admin admin] The token cache is being invalidate because project 6953785191ba4f5a99abad5c6ed914f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:28.838335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a3cee44a-7dfa-4164-a33c-57802a443032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.838965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a3cee44a-7dfa-4164-a33c-57802a443032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:28.840477 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1228/2440] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:28 2022] DELETE /identity/v3/projects/6953785191ba4f5a99abad5c6ed914f1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:28.845373 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5f29d-10e9-42d5-805f-cafacdab1246 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:28.845867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5f29d-10e9-42d5-805f-cafacdab1246 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:28.845989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5f29d-10e9-42d5-805f-cafacdab1246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:28.865824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a7d5f29d-10e9-42d5-805f-cafacdab1246 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:28.867032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a7d5f29d-10e9-42d5-805f-cafacdab1246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:28.934228 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1213/2441] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:28.943957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800929ad-906b-407e-af90-132ceb45c0d1 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:28.944240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800929ad-906b-407e-af90-132ceb45c0d1 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:28.944512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800929ad-906b-407e-af90-132ceb45c0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:28.965550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-800929ad-906b-407e-af90-132ceb45c0d1 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:28.966820 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-800929ad-906b-407e-af90-132ceb45c0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:28.974714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1229/2442] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:28.978989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5f29d-10e9-42d5-805f-cafacdab1246 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:28.988978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5f29d-10e9-42d5-805f-cafacdab1246 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:28.993581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60899c8c-be05-46ec-a902-c674dbb9adbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:28.994236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60899c8c-be05-46ec-a902-c674dbb9adbf admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:28.994524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60899c8c-be05-46ec-a902-c674dbb9adbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:28.994818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60899c8c-be05-46ec-a902-c674dbb9adbf admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:28.995456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60899c8c-be05-46ec-a902-c674dbb9adbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:54:28.995758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60899c8c-be05-46ec-a902-c674dbb9adbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:28.997406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60899c8c-be05-46ec-a902-c674dbb9adbf admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:29.009497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1214/2443] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:54:29.013853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800929ad-906b-407e-af90-132ceb45c0d1 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:29.022930 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800929ad-906b-407e-af90-132ceb45c0d1 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:29.027553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e95d6-2204-475d-9c97-281245a2f48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:29.028144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e95d6-2204-475d-9c97-281245a2f48a admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.028422 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e95d6-2204-475d-9c97-281245a2f48a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.028690 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e95d6-2204-475d-9c97-281245a2f48a admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.029318 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e95d6-2204-475d-9c97-281245a2f48a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:29.031264 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e95d6-2204-475d-9c97-281245a2f48a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:29.057870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1230/2444] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:29 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:29.062040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60899c8c-be05-46ec-a902-c674dbb9adbf admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:29.072555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60899c8c-be05-46ec-a902-c674dbb9adbf admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:29.077124 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:29.130794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804fe96-8f86-45d1-b050-a458d7851c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:29.131393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804fe96-8f86-45d1-b050-a458d7851c64 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.131672 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804fe96-8f86-45d1-b050-a458d7851c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.131944 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804fe96-8f86-45d1-b050-a458d7851c64 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.132585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804fe96-8f86-45d1-b050-a458d7851c64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:29.134580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804fe96-8f86-45d1-b050-a458d7851c64 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:29.155512 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1215/2445] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:29 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:29.159751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e95d6-2204-475d-9c97-281245a2f48a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:29.169979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e95d6-2204-475d-9c97-281245a2f48a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:29.175211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d4bb7-f4c7-449e-99aa-e22b63fc328b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:29.175803 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d4bb7-f4c7-449e-99aa-e22b63fc328b admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.176158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d4bb7-f4c7-449e-99aa-e22b63fc328b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.176430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d4bb7-f4c7-449e-99aa-e22b63fc328b admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.180497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d4bb7-f4c7-449e-99aa-e22b63fc328b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:29.182981 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d4bb7-f4c7-449e-99aa-e22b63fc328b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:29.193341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1231/2446] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:29.197535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804fe96-8f86-45d1-b050-a458d7851c64 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:29.207502 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804fe96-8f86-45d1-b050-a458d7851c64 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:29.212514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fe64a-b2b0-4762-ab08-dbc1a6cb57b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:29.213109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fe64a-b2b0-4762-ab08-dbc1a6cb57b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.213384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fe64a-b2b0-4762-ab08-dbc1a6cb57b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.213655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fe64a-b2b0-4762-ab08-dbc1a6cb57b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.214315 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fe64a-b2b0-4762-ab08-dbc1a6cb57b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:29.215903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fe64a-b2b0-4762-ab08-dbc1a6cb57b5 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:29.230092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1216/2447] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:54:29.234366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d4bb7-f4c7-449e-99aa-e22b63fc328b admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:29.245542 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d4bb7-f4c7-449e-99aa-e22b63fc328b admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:29.252067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c6b07-c08a-45e8-b309-7edbb5b3692a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:29.253420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c6b07-c08a-45e8-b309-7edbb5b3692a admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.253749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c6b07-c08a-45e8-b309-7edbb5b3692a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.254059 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c6b07-c08a-45e8-b309-7edbb5b3692a admin admin] PATH_INFO: `/v3/projects/5cc0bf24251545db973ea273a167a56c/users/20212d2254ee4334be5f3bdb0d864fe6/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.269068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c6b07-c08a-45e8-b309-7edbb5b3692a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc0bf24251545db973ea273a167a56c, user_id=20212d2254ee4334be5f3bdb0d864fe6, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:29.271771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c6b07-c08a-45e8-b309-7edbb5b3692a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:29.284180 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1232/2448] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:29 2022] PUT /identity/v3/projects/5cc0bf24251545db973ea273a167a56c/users/20212d2254ee4334be5f3bdb0d864fe6/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:29.294659 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68177b-6258-4706-9747-228697043068 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.294947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68177b-6258-4706-9747-228697043068 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.295221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68177b-6258-4706-9747-228697043068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.323467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-1c68177b-6258-4706-9747-228697043068 None None] MFA Rules not processed for user `20212d2254ee4334be5f3bdb0d864fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:29.324604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1c68177b-6258-4706-9747-228697043068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:29.387788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1217/2449] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:29.392123 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c6b07-c08a-45e8-b309-7edbb5b3692a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:29.456227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c6b07-c08a-45e8-b309-7edbb5b3692a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:29.461181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2407c-07cd-4a3f-acb1-0bbfe9dc43e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:29.461907 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2407c-07cd-4a3f-acb1-0bbfe9dc43e9 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.462191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2407c-07cd-4a3f-acb1-0bbfe9dc43e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.462463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2407c-07cd-4a3f-acb1-0bbfe9dc43e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.463090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2407c-07cd-4a3f-acb1-0bbfe9dc43e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:29.464998 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2407c-07cd-4a3f-acb1-0bbfe9dc43e9 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:29.493512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1233/2450] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:29 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:29.497597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68177b-6258-4706-9747-228697043068 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:29.508789 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68177b-6258-4706-9747-228697043068 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:29.571669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c675f-927e-4ad0-b974-5f6d518e97a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:29.572382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c675f-927e-4ad0-b974-5f6d518e97a8 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.572663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c675f-927e-4ad0-b974-5f6d518e97a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.572932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c675f-927e-4ad0-b974-5f6d518e97a8 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.573610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c675f-927e-4ad0-b974-5f6d518e97a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:29.575529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c675f-927e-4ad0-b974-5f6d518e97a8 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:29.597241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1218/2451] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:29 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:54:29.601174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2407c-07cd-4a3f-acb1-0bbfe9dc43e9 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:29.611183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2407c-07cd-4a3f-acb1-0bbfe9dc43e9 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:29.616334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb66d6-1ae8-4fe8-b147-8c1c650513b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:29.616920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb66d6-1ae8-4fe8-b147-8c1c650513b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.617196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb66d6-1ae8-4fe8-b147-8c1c650513b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.617468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb66d6-1ae8-4fe8-b147-8c1c650513b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.618138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb66d6-1ae8-4fe8-b147-8c1c650513b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:29.619726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb66d6-1ae8-4fe8-b147-8c1c650513b7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:29.629882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1234/2452] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:29.633783 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c675f-927e-4ad0-b974-5f6d518e97a8 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:29.643649 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c675f-927e-4ad0-b974-5f6d518e97a8 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:29.648359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2d0b9-586d-4fb7-9ba2-1f70a3c56fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:29.649100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2d0b9-586d-4fb7-9ba2-1f70a3c56fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.649328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2d0b9-586d-4fb7-9ba2-1f70a3c56fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.649573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2d0b9-586d-4fb7-9ba2-1f70a3c56fe3 admin admin] PATH_INFO: `/v3/projects/3f5ca5ed013a46c3996ea68c09ac72ce/users/3b6f1380fda54d669e8db047aa9977eb/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.651943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:29.667203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2d0b9-586d-4fb7-9ba2-1f70a3c56fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f5ca5ed013a46c3996ea68c09ac72ce, user_id=3b6f1380fda54d669e8db047aa9977eb, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:29.670690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2d0b9-586d-4fb7-9ba2-1f70a3c56fe3 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:29.684203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1219/2453] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:29 2022] PUT /identity/v3/projects/3f5ca5ed013a46c3996ea68c09ac72ce/users/3b6f1380fda54d669e8db047aa9977eb/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:29.688176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb66d6-1ae8-4fe8-b147-8c1c650513b7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:29.698197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb66d6-1ae8-4fe8-b147-8c1c650513b7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:29.718250 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:29.755570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef96f4b-94a5-49ff-970e-3bdc292eab16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:29.756202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef96f4b-94a5-49ff-970e-3bdc292eab16 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.756484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef96f4b-94a5-49ff-970e-3bdc292eab16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.756759 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef96f4b-94a5-49ff-970e-3bdc292eab16 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.757427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef96f4b-94a5-49ff-970e-3bdc292eab16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:29.759068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef96f4b-94a5-49ff-970e-3bdc292eab16 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:29.769108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1235/2454] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:29.773457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2d0b9-586d-4fb7-9ba2-1f70a3c56fe3 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:29.785007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2d0b9-586d-4fb7-9ba2-1f70a3c56fe3 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:29.789910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c516d5-ae6f-4538-8d86-6b44f784bb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:29.790622 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c516d5-ae6f-4538-8d86-6b44f784bb37 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.790927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c516d5-ae6f-4538-8d86-6b44f784bb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.791221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c516d5-ae6f-4538-8d86-6b44f784bb37 admin admin] PATH_INFO: `/v3/projects/3f5ca5ed013a46c3996ea68c09ac72ce/users/3b6f1380fda54d669e8db047aa9977eb/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.793153 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c516d5-ae6f-4538-8d86-6b44f784bb37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f5ca5ed013a46c3996ea68c09ac72ce, user_id=3b6f1380fda54d669e8db047aa9977eb, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:29.795957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c516d5-ae6f-4538-8d86-6b44f784bb37 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:29.806869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1220/2455] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:29 2022] PUT /identity/v3/projects/3f5ca5ed013a46c3996ea68c09ac72ce/users/3b6f1380fda54d669e8db047aa9977eb/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:29.828813 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32e71a-c0fc-4249-8376-cde4926b8eee None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:29.829038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32e71a-c0fc-4249-8376-cde4926b8eee None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:29.829314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32e71a-c0fc-4249-8376-cde4926b8eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:29.859310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-2e32e71a-c0fc-4249-8376-cde4926b8eee None None] MFA Rules not processed for user `3b6f1380fda54d669e8db047aa9977eb`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:29.860534 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2e32e71a-c0fc-4249-8376-cde4926b8eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:29.936651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1236/2456] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:29.948528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c516d5-ae6f-4538-8d86-6b44f784bb37 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:29.950291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-03c516d5-ae6f-4538-8d86-6b44f784bb37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:30.038043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c516d5-ae6f-4538-8d86-6b44f784bb37 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:30.041868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fb6f7-b6d5-40cc-84fb-ab1d0b621957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:30.042483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fb6f7-b6d5-40cc-84fb-ab1d0b621957 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:30.043254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fb6f7-b6d5-40cc-84fb-ab1d0b621957 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:30.043254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fb6f7-b6d5-40cc-84fb-ab1d0b621957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:30.053976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fb6f7-b6d5-40cc-84fb-ab1d0b621957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:30.055432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fb6f7-b6d5-40cc-84fb-ab1d0b621957 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:30.069860 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1221/2457] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:30.681166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32e71a-c0fc-4249-8376-cde4926b8eee None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:30.825236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32e71a-c0fc-4249-8376-cde4926b8eee None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:30.830830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c997c2-e579-421e-aade-d34e17bdca3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:30.831911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c997c2-e579-421e-aade-d34e17bdca3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:30.832422 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c997c2-e579-421e-aade-d34e17bdca3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:30.833051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c997c2-e579-421e-aade-d34e17bdca3b admin admin] PATH_INFO: `/v3/users/20212d2254ee4334be5f3bdb0d864fe6` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:30.834653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c997c2-e579-421e-aade-d34e17bdca3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=20212d2254ee4334be5f3bdb0d864fe6)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:30.836861 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c997c2-e579-421e-aade-d34e17bdca3b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:30.883136 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fb6f7-b6d5-40cc-84fb-ab1d0b621957 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:30.885515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-110fb6f7-b6d5-40cc-84fb-ab1d0b621957 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:30.907440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-15c997c2-e579-421e-aade-d34e17bdca3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20212d2254ee4334be5f3bdb0d864fe6'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:30.908396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-15c997c2-e579-421e-aade-d34e17bdca3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20212d2254ee4334be5f3bdb0d864fe6'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:30.910581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:30.919921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-15c997c2-e579-421e-aade-d34e17bdca3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20212d2254ee4334be5f3bdb0d864fe6'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:30.920801 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-15c997c2-e579-421e-aade-d34e17bdca3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20212d2254ee4334be5f3bdb0d864fe6'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:30.941891 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-15c997c2-e579-421e-aade-d34e17bdca3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20212d2254ee4334be5f3bdb0d864fe6'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:30.961033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1237/2458] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:30 2022] DELETE /identity/v3/users/20212d2254ee4334be5f3bdb0d864fe6 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:30.965593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105875e-9e4b-40ba-8b63-cec9c18b7224 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:30.965839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105875e-9e4b-40ba-8b63-cec9c18b7224 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:30.966141 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105875e-9e4b-40ba-8b63-cec9c18b7224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:30.989335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-7105875e-9e4b-40ba-8b63-cec9c18b7224 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:30.990623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7105875e-9e4b-40ba-8b63-cec9c18b7224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:31.008414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fb6f7-b6d5-40cc-84fb-ab1d0b621957 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:31.046509 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:31.057042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1238/2459] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:54:31.070063 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105875e-9e4b-40ba-8b63-cec9c18b7224 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:31.071625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7105875e-9e4b-40ba-8b63-cec9c18b7224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:31.079978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8d13d-d024-4c90-a01e-2275a15279bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:31.080717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8d13d-d024-4c90-a01e-2275a15279bd service placement] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:31.081073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8d13d-d024-4c90-a01e-2275a15279bd service placement] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:31.081609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8d13d-d024-4c90-a01e-2275a15279bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:31.084159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f0d8d13d-d024-4c90-a01e-2275a15279bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:31.097109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8d13d-d024-4c90-a01e-2275a15279bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:31.098838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8d13d-d024-4c90-a01e-2275a15279bd service placement] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:31.110209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1222/2460] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:31.133292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105875e-9e4b-40ba-8b63-cec9c18b7224 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:31.137257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b0a5d-89be-420a-a092-cfce586ba229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:31.138097 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b0a5d-89be-420a-a092-cfce586ba229 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:31.138450 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b0a5d-89be-420a-a092-cfce586ba229 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:31.138819 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b0a5d-89be-420a-a092-cfce586ba229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:31.148585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b0a5d-89be-420a-a092-cfce586ba229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:31.150200 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b0a5d-89be-420a-a092-cfce586ba229 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:31.162106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1239/2461] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:54:31.489272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8d13d-d024-4c90-a01e-2275a15279bd service placement] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:31.491174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f0d8d13d-d024-4c90-a01e-2275a15279bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:31.511674 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8d13d-d024-4c90-a01e-2275a15279bd service placement] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:31.516552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:31.517092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:31.517374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:31.517647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] PATH_INFO: `/v3/projects/5cc0bf24251545db973ea273a167a56c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:31.519455 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc0bf24251545db973ea273a167a56c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:31.521310 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:31.571442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc0bf24251545db973ea273a167a56c', 'request_id': 'req-8b8a364f-d5e1-433d-beb9-c711ab5a1429'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:31.578205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc0bf24251545db973ea273a167a56c', 'request_id': 'req-8b8a364f-d5e1-433d-beb9-c711ab5a1429'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:31.584904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc0bf24251545db973ea273a167a56c', 'request_id': 'req-8b8a364f-d5e1-433d-beb9-c711ab5a1429'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:31.597653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] The token cache is being invalidate because project 5cc0bf24251545db973ea273a167a56c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:31.597980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:31.598600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:31.600107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1223/2462] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:31 2022] DELETE /identity/v3/projects/5cc0bf24251545db973ea273a167a56c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:31.604455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b0a5d-89be-420a-a092-cfce586ba229 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:31.614148 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b0a5d-89be-420a-a092-cfce586ba229 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:31.677269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127a528-e455-438a-ab29-959f42d691c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:31.677985 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127a528-e455-438a-ab29-959f42d691c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:31.678264 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127a528-e455-438a-ab29-959f42d691c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:31.678539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127a528-e455-438a-ab29-959f42d691c7 admin admin] PATH_INFO: `/v3/users/3b6f1380fda54d669e8db047aa9977eb` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:31.679675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127a528-e455-438a-ab29-959f42d691c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b6f1380fda54d669e8db047aa9977eb)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:31.681614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127a528-e455-438a-ab29-959f42d691c7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:31.693302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:31.733010 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9127a528-e455-438a-ab29-959f42d691c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b6f1380fda54d669e8db047aa9977eb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:31.733802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9127a528-e455-438a-ab29-959f42d691c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b6f1380fda54d669e8db047aa9977eb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:31.741269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9127a528-e455-438a-ab29-959f42d691c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b6f1380fda54d669e8db047aa9977eb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:31.741800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9127a528-e455-438a-ab29-959f42d691c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b6f1380fda54d669e8db047aa9977eb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:31.755639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9127a528-e455-438a-ab29-959f42d691c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b6f1380fda54d669e8db047aa9977eb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:31.770786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1240/2463] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:31 2022] DELETE /identity/v3/users/3b6f1380fda54d669e8db047aa9977eb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:32.138788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:32.140280 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:32.153734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a364f-d5e1-433d-beb9-c711ab5a1429 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:32.209834 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d35b0-055d-4fed-b934-bd9e82ee57e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:32.210471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d35b0-055d-4fed-b934-bd9e82ee57e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:32.210757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d35b0-055d-4fed-b934-bd9e82ee57e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:32.211033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d35b0-055d-4fed-b934-bd9e82ee57e1 admin admin] PATH_INFO: `/v3/projects/3f5ca5ed013a46c3996ea68c09ac72ce` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:32.212155 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d35b0-055d-4fed-b934-bd9e82ee57e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f5ca5ed013a46c3996ea68c09ac72ce)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:32.214240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60d35b0-055d-4fed-b934-bd9e82ee57e1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:32.261633 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d60d35b0-055d-4fed-b934-bd9e82ee57e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f5ca5ed013a46c3996ea68c09ac72ce', 'request_id': 'req-d60d35b0-055d-4fed-b934-bd9e82ee57e1'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:32.262724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:32.269239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d60d35b0-055d-4fed-b934-bd9e82ee57e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f5ca5ed013a46c3996ea68c09ac72ce', 'request_id': 'req-d60d35b0-055d-4fed-b934-bd9e82ee57e1'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:32.275508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d60d35b0-055d-4fed-b934-bd9e82ee57e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f5ca5ed013a46c3996ea68c09ac72ce', 'request_id': 'req-d60d35b0-055d-4fed-b934-bd9e82ee57e1'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:32.287366 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d60d35b0-055d-4fed-b934-bd9e82ee57e1 admin admin] The token cache is being invalidate because project 3f5ca5ed013a46c3996ea68c09ac72ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:32.287653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d60d35b0-055d-4fed-b934-bd9e82ee57e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:32.288296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d60d35b0-055d-4fed-b934-bd9e82ee57e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:32.289835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1224/2464] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:32 2022] DELETE /identity/v3/projects/3f5ca5ed013a46c3996ea68c09ac72ce => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:32.294831 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5705bc-deb6-41c5-bdfa-571111fb7423 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:32.295291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5705bc-deb6-41c5-bdfa-571111fb7423 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:32.295387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5705bc-deb6-41c5-bdfa-571111fb7423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:32.316094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-fe5705bc-deb6-41c5-bdfa-571111fb7423 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:32.317294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fe5705bc-deb6-41c5-bdfa-571111fb7423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:32.383511 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1241/2465] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:32.392380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88ac95-c034-4e5a-81e1-a7ee3fdebdba None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:32.392667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88ac95-c034-4e5a-81e1-a7ee3fdebdba None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:32.392983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88ac95-c034-4e5a-81e1-a7ee3fdebdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:32.413269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-4e88ac95-c034-4e5a-81e1-a7ee3fdebdba None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:32.414428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4e88ac95-c034-4e5a-81e1-a7ee3fdebdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:32.422428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1225/2466] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:32.427267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5705bc-deb6-41c5-bdfa-571111fb7423 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:32.436815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5705bc-deb6-41c5-bdfa-571111fb7423 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:32.441523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45d1af-8289-4e9e-8fc6-030f5fe43102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:32.442149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45d1af-8289-4e9e-8fc6-030f5fe43102 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:32.442434 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45d1af-8289-4e9e-8fc6-030f5fe43102 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:32.442715 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45d1af-8289-4e9e-8fc6-030f5fe43102 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:32.443386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45d1af-8289-4e9e-8fc6-030f5fe43102 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:54:32.443690 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45d1af-8289-4e9e-8fc6-030f5fe43102 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:32.445295 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45d1af-8289-4e9e-8fc6-030f5fe43102 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:32.456801 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1242/2467] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:54:32.461577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88ac95-c034-4e5a-81e1-a7ee3fdebdba None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:32.471438 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88ac95-c034-4e5a-81e1-a7ee3fdebdba None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:32.476180 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156cc94-2a03-477a-8bfe-2ea51980ecbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:32.476780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156cc94-2a03-477a-8bfe-2ea51980ecbf admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:32.477058 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156cc94-2a03-477a-8bfe-2ea51980ecbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:32.477331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156cc94-2a03-477a-8bfe-2ea51980ecbf admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:32.478005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156cc94-2a03-477a-8bfe-2ea51980ecbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:32.479910 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156cc94-2a03-477a-8bfe-2ea51980ecbf admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:32.510053 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1226/2468] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:32 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:32.520551 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45d1af-8289-4e9e-8fc6-030f5fe43102 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:32.534334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45d1af-8289-4e9e-8fc6-030f5fe43102 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:32.594936 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122b74b-095f-426e-847a-110c0e3d8a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:32.595558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122b74b-095f-426e-847a-110c0e3d8a19 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:32.596316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122b74b-095f-426e-847a-110c0e3d8a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:32.596588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122b74b-095f-426e-847a-110c0e3d8a19 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:32.597220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122b74b-095f-426e-847a-110c0e3d8a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:32.599162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122b74b-095f-426e-847a-110c0e3d8a19 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:32.621417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1243/2469] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:32 2022] POST /identity/v3/users => generated 282 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:32.625450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156cc94-2a03-477a-8bfe-2ea51980ecbf admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:32.636283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156cc94-2a03-477a-8bfe-2ea51980ecbf admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:32.641279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5590f76-a7fa-4826-b158-ef60dab52250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:32.641787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5590f76-a7fa-4826-b158-ef60dab52250 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:32.642142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5590f76-a7fa-4826-b158-ef60dab52250 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:32.642517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5590f76-a7fa-4826-b158-ef60dab52250 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:32.643269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5590f76-a7fa-4826-b158-ef60dab52250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:32.645102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5590f76-a7fa-4826-b158-ef60dab52250 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:32.656349 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1227/2470] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:32.660801 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122b74b-095f-426e-847a-110c0e3d8a19 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:32.667736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:32.671706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122b74b-095f-426e-847a-110c0e3d8a19 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:32.678330 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7263779f-9369-49f2-b422-c78d3ad60d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:32.679137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263779f-9369-49f2-b422-c78d3ad60d5f admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:32.679576 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263779f-9369-49f2-b422-c78d3ad60d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:32.680035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7263779f-9369-49f2-b422-c78d3ad60d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:32.680759 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263779f-9369-49f2-b422-c78d3ad60d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:32.682839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7263779f-9369-49f2-b422-c78d3ad60d5f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:32.693838 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1244/2471] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:32.698059 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5590f76-a7fa-4826-b158-ef60dab52250 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:32.708942 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5590f76-a7fa-4826-b158-ef60dab52250 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:32.713631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731be92-6db1-41ab-bb8a-216eacd3805a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:32.714317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731be92-6db1-41ab-bb8a-216eacd3805a admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:32.714599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731be92-6db1-41ab-bb8a-216eacd3805a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:32.714873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731be92-6db1-41ab-bb8a-216eacd3805a admin admin] PATH_INFO: `/v3/projects/169f027d501140d997e65ec1831bff55/users/326b6fa426d34b6683de07046313e0e5/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:32.730714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731be92-6db1-41ab-bb8a-216eacd3805a admin admin] RBAC: Authorizing `identity:create_grant(project_id=169f027d501140d997e65ec1831bff55, user_id=326b6fa426d34b6683de07046313e0e5, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:32.733388 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731be92-6db1-41ab-bb8a-216eacd3805a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:32.746772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1228/2472] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:32 2022] PUT /identity/v3/projects/169f027d501140d997e65ec1831bff55/users/326b6fa426d34b6683de07046313e0e5/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:54:32.757661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc9aea-c426-422f-a7e2-c0cf52d57186 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:32.757980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc9aea-c426-422f-a7e2-c0cf52d57186 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:32.758261 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc9aea-c426-422f-a7e2-c0cf52d57186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:32.787239 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-ebbc9aea-c426-422f-a7e2-c0cf52d57186 None None] MFA Rules not processed for user `326b6fa426d34b6683de07046313e0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:32.788467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ebbc9aea-c426-422f-a7e2-c0cf52d57186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:32.853059 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1245/2473] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:32.857795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731be92-6db1-41ab-bb8a-216eacd3805a admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:32.925464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731be92-6db1-41ab-bb8a-216eacd3805a admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:32.930301 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8d615-e0a8-4b93-8c4f-20a6ec408b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:32.930918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8d615-e0a8-4b93-8c4f-20a6ec408b1e admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:32.931208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8d615-e0a8-4b93-8c4f-20a6ec408b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:32.931481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8d615-e0a8-4b93-8c4f-20a6ec408b1e admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:32.932128 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8d615-e0a8-4b93-8c4f-20a6ec408b1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:32.934174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8d615-e0a8-4b93-8c4f-20a6ec408b1e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:32.962582 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1229/2474] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:32 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:54:32.966662 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc9aea-c426-422f-a7e2-c0cf52d57186 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:32.978887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc9aea-c426-422f-a7e2-c0cf52d57186 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:33.039684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39643219-61b2-4be4-a03f-5c0638201586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:33.040301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39643219-61b2-4be4-a03f-5c0638201586 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:33.040613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39643219-61b2-4be4-a03f-5c0638201586 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:33.040901 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39643219-61b2-4be4-a03f-5c0638201586 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:33.042033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39643219-61b2-4be4-a03f-5c0638201586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:33.043568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39643219-61b2-4be4-a03f-5c0638201586 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:33.065468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1246/2475] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:32 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:33.071133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8d615-e0a8-4b93-8c4f-20a6ec408b1e admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:33.081580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8d615-e0a8-4b93-8c4f-20a6ec408b1e admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:33.086723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c0ef9-51d8-4d69-8370-fa4bc6c5945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:33.087227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c0ef9-51d8-4d69-8370-fa4bc6c5945d admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:33.087510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c0ef9-51d8-4d69-8370-fa4bc6c5945d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:33.087854 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c0ef9-51d8-4d69-8370-fa4bc6c5945d admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:33.088418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c0ef9-51d8-4d69-8370-fa4bc6c5945d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:33.090058 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c0ef9-51d8-4d69-8370-fa4bc6c5945d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:33.101762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1230/2476] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:33.106213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39643219-61b2-4be4-a03f-5c0638201586 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:33.117059 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39643219-61b2-4be4-a03f-5c0638201586 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:33.125064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec8de8-910f-4da8-832c-45606bf001ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:33.125780 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec8de8-910f-4da8-832c-45606bf001ed admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:33.126073 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec8de8-910f-4da8-832c-45606bf001ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:33.126381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec8de8-910f-4da8-832c-45606bf001ed admin admin] PATH_INFO: `/v3/projects/0149a46e63054b4eacaa2c3dcc380b22/users/36a243253a4b4c0abb5cc8bc47661589/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:33.144946 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec8de8-910f-4da8-832c-45606bf001ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0149a46e63054b4eacaa2c3dcc380b22, user_id=36a243253a4b4c0abb5cc8bc47661589, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:33.147669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ec8de8-910f-4da8-832c-45606bf001ed admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:33.161577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1247/2477] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:33 2022] PUT /identity/v3/projects/0149a46e63054b4eacaa2c3dcc380b22/users/36a243253a4b4c0abb5cc8bc47661589/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:33.165274 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c0ef9-51d8-4d69-8370-fa4bc6c5945d admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:33.176420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c0ef9-51d8-4d69-8370-fa4bc6c5945d admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:33.227140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:33.240572 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55380eb8-780a-49de-bb6d-cedde2a85f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:33.241439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55380eb8-780a-49de-bb6d-cedde2a85f62 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:33.241830 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55380eb8-780a-49de-bb6d-cedde2a85f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:33.242142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55380eb8-780a-49de-bb6d-cedde2a85f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:33.242870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55380eb8-780a-49de-bb6d-cedde2a85f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:33.245247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55380eb8-780a-49de-bb6d-cedde2a85f62 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:33.256358 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1231/2478] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:33.260752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec8de8-910f-4da8-832c-45606bf001ed admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:33.272106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec8de8-910f-4da8-832c-45606bf001ed admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:33.279857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03212cf6-7970-427d-a6d6-9420118acf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:33.281087 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03212cf6-7970-427d-a6d6-9420118acf39 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:33.281997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03212cf6-7970-427d-a6d6-9420118acf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:33.282160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03212cf6-7970-427d-a6d6-9420118acf39 admin admin] PATH_INFO: `/v3/projects/0149a46e63054b4eacaa2c3dcc380b22/users/36a243253a4b4c0abb5cc8bc47661589/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:33.286685 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03212cf6-7970-427d-a6d6-9420118acf39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0149a46e63054b4eacaa2c3dcc380b22, user_id=36a243253a4b4c0abb5cc8bc47661589, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:33.290043 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03212cf6-7970-427d-a6d6-9420118acf39 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:33.309096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1248/2479] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:33 2022] PUT /identity/v3/projects/0149a46e63054b4eacaa2c3dcc380b22/users/36a243253a4b4c0abb5cc8bc47661589/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:33.321466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ec797-39f8-46d1-9e55-d6160477b712 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:33.321789 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ec797-39f8-46d1-9e55-d6160477b712 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:33.322075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ec797-39f8-46d1-9e55-d6160477b712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:33.351863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-966ec797-39f8-46d1-9e55-d6160477b712 None None] MFA Rules not processed for user `36a243253a4b4c0abb5cc8bc47661589`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:33.353303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-966ec797-39f8-46d1-9e55-d6160477b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:33.434188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1232/2480] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:33.442342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03212cf6-7970-427d-a6d6-9420118acf39 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:33.444133 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-03212cf6-7970-427d-a6d6-9420118acf39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:33.488240 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:33.508296 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03212cf6-7970-427d-a6d6-9420118acf39 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:33.512395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66224ce-c02c-4c3d-885c-29aaf15ee2ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:33.513003 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66224ce-c02c-4c3d-885c-29aaf15ee2ef service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:33.513282 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66224ce-c02c-4c3d-885c-29aaf15ee2ef service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:33.513548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66224ce-c02c-4c3d-885c-29aaf15ee2ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:33.523623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66224ce-c02c-4c3d-885c-29aaf15ee2ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:33.525629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66224ce-c02c-4c3d-885c-29aaf15ee2ef service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:33.535463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1249/2481] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:33.957807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ec797-39f8-46d1-9e55-d6160477b712 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:34.024362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ec797-39f8-46d1-9e55-d6160477b712 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:34.030299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:34.031001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:34.031287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:34.031566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] PATH_INFO: `/v3/users/326b6fa426d34b6683de07046313e0e5` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:34.032734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=326b6fa426d34b6683de07046313e0e5)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:34.035453 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:34.084220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326b6fa426d34b6683de07046313e0e5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.084864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326b6fa426d34b6683de07046313e0e5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.098534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326b6fa426d34b6683de07046313e0e5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.104300 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:34.106519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326b6fa426d34b6683de07046313e0e5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.107335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326b6fa426d34b6683de07046313e0e5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.122106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1233/2482] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:33 2022] DELETE /identity/v3/users/326b6fa426d34b6683de07046313e0e5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:34.126718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f13ac6-8656-43e5-b317-0b074528b11b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:34.127121 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f13ac6-8656-43e5-b317-0b074528b11b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:34.127453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f13ac6-8656-43e5-b317-0b074528b11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:34.146038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d8f13ac6-8656-43e5-b317-0b074528b11b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:34.147337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d8f13ac6-8656-43e5-b317-0b074528b11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:34.207206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1250/2483] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:54:34.217470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:34.219003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:34.280567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648558f-8d01-4557-a0c6-86a8bdef7454 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:34.284991 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4128d-01aa-4b04-a55d-9ba08ac6f646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:34.285621 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4128d-01aa-4b04-a55d-9ba08ac6f646 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:34.285940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4128d-01aa-4b04-a55d-9ba08ac6f646 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:34.286237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4128d-01aa-4b04-a55d-9ba08ac6f646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:34.296085 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4128d-01aa-4b04-a55d-9ba08ac6f646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:34.297546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4128d-01aa-4b04-a55d-9ba08ac6f646 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:34.308080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1234/2484] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:34.631031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82bf86-ffb3-4620-bc78-e2d767b25805 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:34.631569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82bf86-ffb3-4620-bc78-e2d767b25805 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:34.631569 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82bf86-ffb3-4620-bc78-e2d767b25805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:34.650912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-9c82bf86-ffb3-4620-bc78-e2d767b25805 None None] MFA Rules not processed for user `3c0b7528b0ff4b09923a0e951d052ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:34.652117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9c82bf86-ffb3-4620-bc78-e2d767b25805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:34.692182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4128d-01aa-4b04-a55d-9ba08ac6f646 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:34.693665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2ee4128d-01aa-4b04-a55d-9ba08ac6f646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:34.708210 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4128d-01aa-4b04-a55d-9ba08ac6f646 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:34.713163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:34.713842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:34.714136 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:34.714393 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] PATH_INFO: `/v3/projects/169f027d501140d997e65ec1831bff55` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:34.716033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=169f027d501140d997e65ec1831bff55)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:34.718200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:34.742685 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1251/2485] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:34.770579 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '169f027d501140d997e65ec1831bff55', 'request_id': 'req-584351ab-8d61-4800-8b34-47f2d8d40bdf'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.777617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '169f027d501140d997e65ec1831bff55', 'request_id': 'req-584351ab-8d61-4800-8b34-47f2d8d40bdf'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.783750 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '169f027d501140d997e65ec1831bff55', 'request_id': 'req-584351ab-8d61-4800-8b34-47f2d8d40bdf'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.796627 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] The token cache is being invalidate because project 169f027d501140d997e65ec1831bff55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:34.796987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.797757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.799863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1235/2486] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:34 2022] DELETE /identity/v3/projects/169f027d501140d997e65ec1831bff55 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:54:34.803870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82bf86-ffb3-4620-bc78-e2d767b25805 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:34.814941 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82bf86-ffb3-4620-bc78-e2d767b25805 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:34.874043 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a60f55-cd1a-4d14-928a-e1387d2edd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:34.874865 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a60f55-cd1a-4d14-928a-e1387d2edd8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:34.875345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a60f55-cd1a-4d14-928a-e1387d2edd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:34.875621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a60f55-cd1a-4d14-928a-e1387d2edd8f admin admin] PATH_INFO: `/v3/users/36a243253a4b4c0abb5cc8bc47661589` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:34.877655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a60f55-cd1a-4d14-928a-e1387d2edd8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=36a243253a4b4c0abb5cc8bc47661589)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:34.879960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a60f55-cd1a-4d14-928a-e1387d2edd8f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:34.911562 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:34.933149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a2a60f55-cd1a-4d14-928a-e1387d2edd8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36a243253a4b4c0abb5cc8bc47661589'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.933892 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a2a60f55-cd1a-4d14-928a-e1387d2edd8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36a243253a4b4c0abb5cc8bc47661589'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.941884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a2a60f55-cd1a-4d14-928a-e1387d2edd8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36a243253a4b4c0abb5cc8bc47661589'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.942699 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a2a60f55-cd1a-4d14-928a-e1387d2edd8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36a243253a4b4c0abb5cc8bc47661589'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.956026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a2a60f55-cd1a-4d14-928a-e1387d2edd8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36a243253a4b4c0abb5cc8bc47661589'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:34.969370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1252/2487] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:34 2022] DELETE /identity/v3/users/36a243253a4b4c0abb5cc8bc47661589 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:35.260588 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:35.262399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:35.286888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584351ab-8d61-4800-8b34-47f2d8d40bdf admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:35.354719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca6a2e-8b87-48b6-a637-8171d9684a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:35.355348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca6a2e-8b87-48b6-a637-8171d9684a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:35.355654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca6a2e-8b87-48b6-a637-8171d9684a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:35.355924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca6a2e-8b87-48b6-a637-8171d9684a1a admin admin] PATH_INFO: `/v3/projects/0149a46e63054b4eacaa2c3dcc380b22` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:35.357031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca6a2e-8b87-48b6-a637-8171d9684a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0149a46e63054b4eacaa2c3dcc380b22)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:35.358987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca6a2e-8b87-48b6-a637-8171d9684a1a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:35.378983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:35.419977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-44ca6a2e-8b87-48b6-a637-8171d9684a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0149a46e63054b4eacaa2c3dcc380b22', 'request_id': 'req-44ca6a2e-8b87-48b6-a637-8171d9684a1a'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:35.427046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-44ca6a2e-8b87-48b6-a637-8171d9684a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0149a46e63054b4eacaa2c3dcc380b22', 'request_id': 'req-44ca6a2e-8b87-48b6-a637-8171d9684a1a'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:35.433776 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-44ca6a2e-8b87-48b6-a637-8171d9684a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0149a46e63054b4eacaa2c3dcc380b22', 'request_id': 'req-44ca6a2e-8b87-48b6-a637-8171d9684a1a'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:35.453964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-44ca6a2e-8b87-48b6-a637-8171d9684a1a admin admin] The token cache is being invalidate because project 0149a46e63054b4eacaa2c3dcc380b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:35.454304 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-44ca6a2e-8b87-48b6-a637-8171d9684a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:35.455344 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-44ca6a2e-8b87-48b6-a637-8171d9684a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:35.469833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1236/2488] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:35 2022] DELETE /identity/v3/projects/0149a46e63054b4eacaa2c3dcc380b22 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:35.470587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c8881-6cd3-4705-91a0-c34d4fd3250e None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:35.470587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c8881-6cd3-4705-91a0-c34d4fd3250e None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:35.470587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c8881-6cd3-4705-91a0-c34d4fd3250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:35.492826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-061c8881-6cd3-4705-91a0-c34d4fd3250e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:35.494082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-061c8881-6cd3-4705-91a0-c34d4fd3250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:35.554812 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1253/2489] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:35.562505 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a74aa5-3c47-4adb-850f-3f609f0830cc None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:35.562817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a74aa5-3c47-4adb-850f-3f609f0830cc None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:35.563091 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a74aa5-3c47-4adb-850f-3f609f0830cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:35.585052 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-81a74aa5-3c47-4adb-850f-3f609f0830cc None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:35.586406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-81a74aa5-3c47-4adb-850f-3f609f0830cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:35.594934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1237/2490] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:35.599822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c8881-6cd3-4705-91a0-c34d4fd3250e None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:35.612935 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c8881-6cd3-4705-91a0-c34d4fd3250e None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:35.618248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be123ca2-da87-4f7c-b091-ea34d5085d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:35.618849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be123ca2-da87-4f7c-b091-ea34d5085d46 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:35.619151 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be123ca2-da87-4f7c-b091-ea34d5085d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:35.619423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be123ca2-da87-4f7c-b091-ea34d5085d46 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:35.620075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be123ca2-da87-4f7c-b091-ea34d5085d46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:54:35.620376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be123ca2-da87-4f7c-b091-ea34d5085d46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:35.622557 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be123ca2-da87-4f7c-b091-ea34d5085d46 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:35.635643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1254/2491] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:54:35.639938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a74aa5-3c47-4adb-850f-3f609f0830cc None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:35.649322 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a74aa5-3c47-4adb-850f-3f609f0830cc None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:35.654501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555be785-ca7e-455a-9361-9481129b75f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:35.655115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555be785-ca7e-455a-9361-9481129b75f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:35.655389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555be785-ca7e-455a-9361-9481129b75f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:35.655655 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555be785-ca7e-455a-9361-9481129b75f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:35.656304 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555be785-ca7e-455a-9361-9481129b75f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:35.659266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555be785-ca7e-455a-9361-9481129b75f2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:35.694705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1238/2492] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:35 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:35.698897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be123ca2-da87-4f7c-b091-ea34d5085d46 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:35.710383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be123ca2-da87-4f7c-b091-ea34d5085d46 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:35.782038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf24109-0d23-4aa1-9840-6938255106db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:35.782756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf24109-0d23-4aa1-9840-6938255106db admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:35.783089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf24109-0d23-4aa1-9840-6938255106db admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:35.784000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf24109-0d23-4aa1-9840-6938255106db admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:35.784669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf24109-0d23-4aa1-9840-6938255106db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:35.786868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf24109-0d23-4aa1-9840-6938255106db admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:35.810811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1255/2493] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:35 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:35.814969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555be785-ca7e-455a-9361-9481129b75f2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:35.825177 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555be785-ca7e-455a-9361-9481129b75f2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:35.829907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5e8b2-d699-4539-ae34-77b5d35cfced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:35.831628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5e8b2-d699-4539-ae34-77b5d35cfced admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:35.831915 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5e8b2-d699-4539-ae34-77b5d35cfced admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:35.832190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5e8b2-d699-4539-ae34-77b5d35cfced admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:35.832836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5e8b2-d699-4539-ae34-77b5d35cfced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:35.834744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5e8b2-d699-4539-ae34-77b5d35cfced admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:35.845562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1239/2494] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:35.850207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf24109-0d23-4aa1-9840-6938255106db admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:35.861628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf24109-0d23-4aa1-9840-6938255106db admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:35.866663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba21fb-c669-4e54-87f6-6db07a070ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:35.867304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba21fb-c669-4e54-87f6-6db07a070ddc admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:35.867588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba21fb-c669-4e54-87f6-6db07a070ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:35.867863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba21fb-c669-4e54-87f6-6db07a070ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:35.868508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba21fb-c669-4e54-87f6-6db07a070ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:35.870164 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba21fb-c669-4e54-87f6-6db07a070ddc admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:35.881215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1256/2495] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:35.885080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5e8b2-d699-4539-ae34-77b5d35cfced admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:35.895877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5e8b2-d699-4539-ae34-77b5d35cfced admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:35.901235 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927e7e3-de71-4fd7-b83a-49e7528b7368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:35.902036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927e7e3-de71-4fd7-b83a-49e7528b7368 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:35.902539 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927e7e3-de71-4fd7-b83a-49e7528b7368 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:35.902777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927e7e3-de71-4fd7-b83a-49e7528b7368 admin admin] PATH_INFO: `/v3/projects/8800ca6f84c141848f725d2945f06001/users/9b28af6b53964e2a88ed0eed2d19b100/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:35.918661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927e7e3-de71-4fd7-b83a-49e7528b7368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8800ca6f84c141848f725d2945f06001, user_id=9b28af6b53964e2a88ed0eed2d19b100, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:35.921616 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927e7e3-de71-4fd7-b83a-49e7528b7368 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:35.935484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1240/2496] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:35 2022] PUT /identity/v3/projects/8800ca6f84c141848f725d2945f06001/users/9b28af6b53964e2a88ed0eed2d19b100/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:35.946013 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1bb67-270a-43aa-8981-93485eb40c92 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:35.946342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1bb67-270a-43aa-8981-93485eb40c92 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:35.946634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1bb67-270a-43aa-8981-93485eb40c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:35.966938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:35.979007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-30c1bb67-270a-43aa-8981-93485eb40c92 None None] MFA Rules not processed for user `9b28af6b53964e2a88ed0eed2d19b100`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:35.980226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-30c1bb67-270a-43aa-8981-93485eb40c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:36.045984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1257/2497] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:36.050309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927e7e3-de71-4fd7-b83a-49e7528b7368 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:36.121220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927e7e3-de71-4fd7-b83a-49e7528b7368 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:36.126172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5287b4-3f1d-458b-8e8d-2784bac90247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:36.126664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5287b4-3f1d-458b-8e8d-2784bac90247 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:36.127429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5287b4-3f1d-458b-8e8d-2784bac90247 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:36.127701 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5287b4-3f1d-458b-8e8d-2784bac90247 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:36.128338 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5287b4-3f1d-458b-8e8d-2784bac90247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:36.130262 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5287b4-3f1d-458b-8e8d-2784bac90247 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:36.161132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1241/2498] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:36 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:36.163008 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1bb67-270a-43aa-8981-93485eb40c92 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:36.173798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1bb67-270a-43aa-8981-93485eb40c92 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:36.231309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f0b35-98ee-4d26-8f98-95032717ebb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:36.231913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f0b35-98ee-4d26-8f98-95032717ebb2 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:36.232192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f0b35-98ee-4d26-8f98-95032717ebb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:36.232459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f0b35-98ee-4d26-8f98-95032717ebb2 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:36.233087 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f0b35-98ee-4d26-8f98-95032717ebb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:36.235115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f0b35-98ee-4d26-8f98-95032717ebb2 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:36.256034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1258/2499] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:36 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:36.260054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5287b4-3f1d-458b-8e8d-2784bac90247 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:36.270908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5287b4-3f1d-458b-8e8d-2784bac90247 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:36.275475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c5e0c-45e0-45d4-b01a-3da497fdd78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:36.276067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c5e0c-45e0-45d4-b01a-3da497fdd78c admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:36.276343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c5e0c-45e0-45d4-b01a-3da497fdd78c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:36.276611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c5e0c-45e0-45d4-b01a-3da497fdd78c admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:36.277242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c5e0c-45e0-45d4-b01a-3da497fdd78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:36.278858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c5e0c-45e0-45d4-b01a-3da497fdd78c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:36.289292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1242/2500] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:36.293226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f0b35-98ee-4d26-8f98-95032717ebb2 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:36.303091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f0b35-98ee-4d26-8f98-95032717ebb2 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:36.307508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d99aa6-f49e-496f-a424-d3dd126acd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:36.308150 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d99aa6-f49e-496f-a424-d3dd126acd64 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:36.308425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d99aa6-f49e-496f-a424-d3dd126acd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:36.308695 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d99aa6-f49e-496f-a424-d3dd126acd64 admin admin] PATH_INFO: `/v3/projects/b4f372f6293a4e638ca5d9d95162ff86/users/ca3ce21cadd34eaf8fc6c14811754ef0/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:36.323717 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d99aa6-f49e-496f-a424-d3dd126acd64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f372f6293a4e638ca5d9d95162ff86, user_id=ca3ce21cadd34eaf8fc6c14811754ef0, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:36.326470 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d99aa6-f49e-496f-a424-d3dd126acd64 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:36.340128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1259/2501] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:36 2022] PUT /identity/v3/projects/b4f372f6293a4e638ca5d9d95162ff86/users/ca3ce21cadd34eaf8fc6c14811754ef0/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:36.344032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c5e0c-45e0-45d4-b01a-3da497fdd78c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:36.353942 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c5e0c-45e0-45d4-b01a-3da497fdd78c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:36.365559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:36.412141 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf4bbb-0074-46d3-8a40-e4cca1e639c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:36.412734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf4bbb-0074-46d3-8a40-e4cca1e639c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:36.413012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf4bbb-0074-46d3-8a40-e4cca1e639c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:36.413283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf4bbb-0074-46d3-8a40-e4cca1e639c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:36.413938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf4bbb-0074-46d3-8a40-e4cca1e639c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:36.415528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf4bbb-0074-46d3-8a40-e4cca1e639c7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:36.425493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1243/2502] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:54:36.429593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d99aa6-f49e-496f-a424-d3dd126acd64 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:36.440313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d99aa6-f49e-496f-a424-d3dd126acd64 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:36.444810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfc551-73b3-4093-9585-4cf0239bc600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:36.445473 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfc551-73b3-4093-9585-4cf0239bc600 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:36.445792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfc551-73b3-4093-9585-4cf0239bc600 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:36.446069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfc551-73b3-4093-9585-4cf0239bc600 admin admin] PATH_INFO: `/v3/projects/b4f372f6293a4e638ca5d9d95162ff86/users/ca3ce21cadd34eaf8fc6c14811754ef0/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:36.448025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfc551-73b3-4093-9585-4cf0239bc600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f372f6293a4e638ca5d9d95162ff86, user_id=ca3ce21cadd34eaf8fc6c14811754ef0, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:36.450711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfc551-73b3-4093-9585-4cf0239bc600 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:36.461881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1260/2503] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:36 2022] PUT /identity/v3/projects/b4f372f6293a4e638ca5d9d95162ff86/users/ca3ce21cadd34eaf8fc6c14811754ef0/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:36.530726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafa83b-1005-48ed-95cc-7c8fa6f21baf None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:36.531014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafa83b-1005-48ed-95cc-7c8fa6f21baf None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:36.531297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafa83b-1005-48ed-95cc-7c8fa6f21baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:36.562085 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-baafa83b-1005-48ed-95cc-7c8fa6f21baf None None] MFA Rules not processed for user `ca3ce21cadd34eaf8fc6c14811754ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:36.563301 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-baafa83b-1005-48ed-95cc-7c8fa6f21baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:36.640763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1244/2504] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:36 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:36.653676 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfc551-73b3-4093-9585-4cf0239bc600 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:36.657313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-abcfc551-73b3-4093-9585-4cf0239bc600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:36.719228 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfc551-73b3-4093-9585-4cf0239bc600 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:36.723161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043fe56-44ea-4d86-8ded-73ec7b1e32ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:36.723774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043fe56-44ea-4d86-8ded-73ec7b1e32ee service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:36.724062 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043fe56-44ea-4d86-8ded-73ec7b1e32ee service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:36.724369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043fe56-44ea-4d86-8ded-73ec7b1e32ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:36.731145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:36.737565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043fe56-44ea-4d86-8ded-73ec7b1e32ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:36.739158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043fe56-44ea-4d86-8ded-73ec7b1e32ee service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:36.750664 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1261/2505] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:36 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:36.800892 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafa83b-1005-48ed-95cc-7c8fa6f21baf None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:36.802318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-baafa83b-1005-48ed-95cc-7c8fa6f21baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:36.815621 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafa83b-1005-48ed-95cc-7c8fa6f21baf None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:36.819435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d6fbc-2af9-4fc6-a471-dc2175cdeaa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:36.820037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d6fbc-2af9-4fc6-a471-dc2175cdeaa0 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:36.820443 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d6fbc-2af9-4fc6-a471-dc2175cdeaa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:36.820719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d6fbc-2af9-4fc6-a471-dc2175cdeaa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:36.880543 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d6fbc-2af9-4fc6-a471-dc2175cdeaa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:36.882034 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d6fbc-2af9-4fc6-a471-dc2175cdeaa0 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:36.891007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1245/2506] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:40.265090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043fe56-44ea-4d86-8ded-73ec7b1e32ee service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:40.324942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043fe56-44ea-4d86-8ded-73ec7b1e32ee service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:40.329465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:40.330621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:40.331130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:40.331403 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] PATH_INFO: `/v3/users/9b28af6b53964e2a88ed0eed2d19b100` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:40.332507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b28af6b53964e2a88ed0eed2d19b100)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:40.334742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:40.381284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b28af6b53964e2a88ed0eed2d19b100'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:40.381985 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b28af6b53964e2a88ed0eed2d19b100'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:40.388850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b28af6b53964e2a88ed0eed2d19b100'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:40.389476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b28af6b53964e2a88ed0eed2d19b100'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:40.401890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b28af6b53964e2a88ed0eed2d19b100'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:40.414500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1262/2507] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:40 2022] DELETE /identity/v3/users/9b28af6b53964e2a88ed0eed2d19b100 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:40.418979 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428bafd-3e1e-476e-94f1-5a0ab270e383 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:40.419467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428bafd-3e1e-476e-94f1-5a0ab270e383 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:40.419569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428bafd-3e1e-476e-94f1-5a0ab270e383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:40.440772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-5428bafd-3e1e-476e-94f1-5a0ab270e383 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:40.442006 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5428bafd-3e1e-476e-94f1-5a0ab270e383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:40.471336 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:40.500873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1246/2508] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:40.511108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:40.512578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:40.570754 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fa85a-370e-410f-8fce-45154aeae466 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:40.574336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e26a9c-9435-4ba4-930c-4b4358c2d5de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:40.574938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e26a9c-9435-4ba4-930c-4b4358c2d5de service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:40.575216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e26a9c-9435-4ba4-930c-4b4358c2d5de service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:40.575486 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e26a9c-9435-4ba4-930c-4b4358c2d5de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:40.585181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e26a9c-9435-4ba4-930c-4b4358c2d5de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:40.586645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e26a9c-9435-4ba4-930c-4b4358c2d5de service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:40.596572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1263/2509] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:40.872800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428bafd-3e1e-476e-94f1-5a0ab270e383 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:40.874369 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5428bafd-3e1e-476e-94f1-5a0ab270e383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:40.886506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428bafd-3e1e-476e-94f1-5a0ab270e383 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:40.892147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:40.892779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:40.893061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:40.893331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] PATH_INFO: `/v3/projects/8800ca6f84c141848f725d2945f06001` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:40.894547 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8800ca6f84c141848f725d2945f06001)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:40.896562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:40.943337 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8800ca6f84c141848f725d2945f06001', 'request_id': 'req-3fda7058-f60f-4e27-8440-8290b2b2c340'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:40.949380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8800ca6f84c141848f725d2945f06001', 'request_id': 'req-3fda7058-f60f-4e27-8440-8290b2b2c340'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:40.955448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8800ca6f84c141848f725d2945f06001', 'request_id': 'req-3fda7058-f60f-4e27-8440-8290b2b2c340'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:40.968630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] The token cache is being invalidate because project 8800ca6f84c141848f725d2945f06001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:40.968916 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:40.969581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:40.971101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1247/2510] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:40 2022] DELETE /identity/v3/projects/8800ca6f84c141848f725d2945f06001 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:40.975002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e26a9c-9435-4ba4-930c-4b4358c2d5de service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:40.985218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e26a9c-9435-4ba4-930c-4b4358c2d5de service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:40.996666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:41.041863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec7287-0e4e-4c6e-9d5c-1bf0894815b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:41.042489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec7287-0e4e-4c6e-9d5c-1bf0894815b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:41.042769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec7287-0e4e-4c6e-9d5c-1bf0894815b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:41.043050 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec7287-0e4e-4c6e-9d5c-1bf0894815b0 admin admin] PATH_INFO: `/v3/users/ca3ce21cadd34eaf8fc6c14811754ef0` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:41.044149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec7287-0e4e-4c6e-9d5c-1bf0894815b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca3ce21cadd34eaf8fc6c14811754ef0)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:41.046168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec7287-0e4e-4c6e-9d5c-1bf0894815b0 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:41.090201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-41ec7287-0e4e-4c6e-9d5c-1bf0894815b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca3ce21cadd34eaf8fc6c14811754ef0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:41.090856 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-41ec7287-0e4e-4c6e-9d5c-1bf0894815b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca3ce21cadd34eaf8fc6c14811754ef0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:41.097799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-41ec7287-0e4e-4c6e-9d5c-1bf0894815b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca3ce21cadd34eaf8fc6c14811754ef0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:41.098429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-41ec7287-0e4e-4c6e-9d5c-1bf0894815b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca3ce21cadd34eaf8fc6c14811754ef0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:41.109984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-41ec7287-0e4e-4c6e-9d5c-1bf0894815b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca3ce21cadd34eaf8fc6c14811754ef0'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:41.122655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1264/2511] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:40 2022] DELETE /identity/v3/users/ca3ce21cadd34eaf8fc6c14811754ef0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:41.415424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:41.416857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:41.429636 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda7058-f60f-4e27-8440-8290b2b2c340 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:41.487175 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:41.487813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:41.488095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:41.488369 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138 admin admin] PATH_INFO: `/v3/projects/b4f372f6293a4e638ca5d9d95162ff86` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:41.489482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4f372f6293a4e638ca5d9d95162ff86)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:41.491442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:41.536574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4f372f6293a4e638ca5d9d95162ff86', 'request_id': 'req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:41.542436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4f372f6293a4e638ca5d9d95162ff86', 'request_id': 'req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:41.546927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:41.548156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4f372f6293a4e638ca5d9d95162ff86', 'request_id': 'req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:41.561119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138 admin admin] The token cache is being invalidate because project b4f372f6293a4e638ca5d9d95162ff86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:41.561411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:41.562048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3d0f8ca7-7efb-4b6a-a69d-d04823b5b138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:41.563505 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1248/2512] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:41 2022] DELETE /identity/v3/projects/b4f372f6293a4e638ca5d9d95162ff86 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:41.569017 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daf29b-66ca-4137-8db8-03e82b00041f None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:41.569512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daf29b-66ca-4137-8db8-03e82b00041f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:41.569636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daf29b-66ca-4137-8db8-03e82b00041f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:41.590588 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-78daf29b-66ca-4137-8db8-03e82b00041f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:41.591769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-78daf29b-66ca-4137-8db8-03e82b00041f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:41.650321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1265/2513] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:41.658200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a5239-e482-44e9-8ab0-7a9d9db558dc None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:41.658529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a5239-e482-44e9-8ab0-7a9d9db558dc None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:41.658825 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a5239-e482-44e9-8ab0-7a9d9db558dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:41.679001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-e06a5239-e482-44e9-8ab0-7a9d9db558dc None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:41.680198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e06a5239-e482-44e9-8ab0-7a9d9db558dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:41.688326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1249/2514] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:41.692932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78daf29b-66ca-4137-8db8-03e82b00041f None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:41.703068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78daf29b-66ca-4137-8db8-03e82b00041f None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:41.708540 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908d63c-d72e-44a7-8201-f3224165642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:41.709139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908d63c-d72e-44a7-8201-f3224165642d admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:41.709417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908d63c-d72e-44a7-8201-f3224165642d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:41.709712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908d63c-d72e-44a7-8201-f3224165642d admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:41.710366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908d63c-d72e-44a7-8201-f3224165642d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:54:41.710668 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908d63c-d72e-44a7-8201-f3224165642d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:41.712262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908d63c-d72e-44a7-8201-f3224165642d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:41.727881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1266/2515] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:54:41.732501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a5239-e482-44e9-8ab0-7a9d9db558dc None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:41.742387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a5239-e482-44e9-8ab0-7a9d9db558dc None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:41.747075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e30a3-a29a-4606-b2e9-f5a23d2abe8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:41.747677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e30a3-a29a-4606-b2e9-f5a23d2abe8b admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:41.747958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e30a3-a29a-4606-b2e9-f5a23d2abe8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:41.748228 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e30a3-a29a-4606-b2e9-f5a23d2abe8b admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:41.748854 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e30a3-a29a-4606-b2e9-f5a23d2abe8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:41.751008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8e30a3-a29a-4606-b2e9-f5a23d2abe8b admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:41.779473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1250/2516] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:41 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:54:41.783822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908d63c-d72e-44a7-8201-f3224165642d admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:41.794064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908d63c-d72e-44a7-8201-f3224165642d admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:41.813206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:41.854809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4446777-6acd-4c6c-8749-851da7501d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:41.855413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4446777-6acd-4c6c-8749-851da7501d9f admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:41.855700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4446777-6acd-4c6c-8749-851da7501d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:41.856011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4446777-6acd-4c6c-8749-851da7501d9f admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:41.856641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4446777-6acd-4c6c-8749-851da7501d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:41.858797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4446777-6acd-4c6c-8749-851da7501d9f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:41.880538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1267/2517] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:41 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:41.884786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e30a3-a29a-4606-b2e9-f5a23d2abe8b admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:41.894906 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e30a3-a29a-4606-b2e9-f5a23d2abe8b admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:41.899471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eff120-cb68-40d6-a665-64ea6f490d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:41.900061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eff120-cb68-40d6-a665-64ea6f490d3e admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:41.900340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eff120-cb68-40d6-a665-64ea6f490d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:41.900611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eff120-cb68-40d6-a665-64ea6f490d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:41.901241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eff120-cb68-40d6-a665-64ea6f490d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:41.902868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eff120-cb68-40d6-a665-64ea6f490d3e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:41.912475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1251/2518] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:41.916101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4446777-6acd-4c6c-8749-851da7501d9f admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:41.926084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4446777-6acd-4c6c-8749-851da7501d9f admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:41.930597 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d6a51-f986-4377-9f5f-b696937fa203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:41.931183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d6a51-f986-4377-9f5f-b696937fa203 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:41.931458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d6a51-f986-4377-9f5f-b696937fa203 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:41.931728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d6a51-f986-4377-9f5f-b696937fa203 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:41.932354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d6a51-f986-4377-9f5f-b696937fa203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:41.933983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d6a51-f986-4377-9f5f-b696937fa203 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:41.944697 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1268/2519] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:41.948559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eff120-cb68-40d6-a665-64ea6f490d3e admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:41.958672 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eff120-cb68-40d6-a665-64ea6f490d3e admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:41.963274 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fa175-c5e7-4b26-a858-21e05c91b159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:41.963905 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fa175-c5e7-4b26-a858-21e05c91b159 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:41.964186 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fa175-c5e7-4b26-a858-21e05c91b159 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:41.964462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54fa175-c5e7-4b26-a858-21e05c91b159 admin admin] PATH_INFO: `/v3/projects/3672584ef1f8479ea0829826e5906999/users/3682a93a85624f558ba76630961ca67b/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:41.979279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fa175-c5e7-4b26-a858-21e05c91b159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3672584ef1f8479ea0829826e5906999, user_id=3682a93a85624f558ba76630961ca67b, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:41.981986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54fa175-c5e7-4b26-a858-21e05c91b159 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:41.994993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1252/2520] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:41 2022] PUT /identity/v3/projects/3672584ef1f8479ea0829826e5906999/users/3682a93a85624f558ba76630961ca67b/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:42.005575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0c1e2-8205-4b0e-8088-fccb68f8b7e8 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.005915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0c1e2-8205-4b0e-8088-fccb68f8b7e8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.006192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0c1e2-8205-4b0e-8088-fccb68f8b7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.032617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-6ba0c1e2-8205-4b0e-8088-fccb68f8b7e8 None None] MFA Rules not processed for user `3682a93a85624f558ba76630961ca67b`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:42.033847 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6ba0c1e2-8205-4b0e-8088-fccb68f8b7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:42.096530 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1269/2521] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:42.100971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fa175-c5e7-4b26-a858-21e05c91b159 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:42.160846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54fa175-c5e7-4b26-a858-21e05c91b159 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:42.165710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5651243-b0ac-4d57-81b2-e6d82c9769f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:42.166317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5651243-b0ac-4d57-81b2-e6d82c9769f6 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.166594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5651243-b0ac-4d57-81b2-e6d82c9769f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.166863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5651243-b0ac-4d57-81b2-e6d82c9769f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.167493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5651243-b0ac-4d57-81b2-e6d82c9769f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:42.169376 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5651243-b0ac-4d57-81b2-e6d82c9769f6 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:42.195405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1253/2522] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:42 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:42.199618 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0c1e2-8205-4b0e-8088-fccb68f8b7e8 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:42.211391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0c1e2-8205-4b0e-8088-fccb68f8b7e8 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:42.266191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657c11f-dd3e-40c4-8140-1e6f10149571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:42.266788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657c11f-dd3e-40c4-8140-1e6f10149571 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.267068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657c11f-dd3e-40c4-8140-1e6f10149571 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.267414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657c11f-dd3e-40c4-8140-1e6f10149571 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.268032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657c11f-dd3e-40c4-8140-1e6f10149571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:42.269952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657c11f-dd3e-40c4-8140-1e6f10149571 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:42.290476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1270/2523] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:42 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:42.294589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5651243-b0ac-4d57-81b2-e6d82c9769f6 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:42.305371 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5651243-b0ac-4d57-81b2-e6d82c9769f6 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:42.309950 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443888dc-db2a-4191-9953-cc3415061e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:42.310533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443888dc-db2a-4191-9953-cc3415061e0e admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.310810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443888dc-db2a-4191-9953-cc3415061e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.311080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443888dc-db2a-4191-9953-cc3415061e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.311711 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443888dc-db2a-4191-9953-cc3415061e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:42.313293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443888dc-db2a-4191-9953-cc3415061e0e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:42.323624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1254/2524] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:42.327484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657c11f-dd3e-40c4-8140-1e6f10149571 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:42.337896 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657c11f-dd3e-40c4-8140-1e6f10149571 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:42.342498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2c397-f331-4c39-9501-8d005cc0dacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:42.343087 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2c397-f331-4c39-9501-8d005cc0dacc admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.343362 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2c397-f331-4c39-9501-8d005cc0dacc admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.343631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2c397-f331-4c39-9501-8d005cc0dacc admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.344263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2c397-f331-4c39-9501-8d005cc0dacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:42.346411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2c397-f331-4c39-9501-8d005cc0dacc admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:42.356152 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1271/2525] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:42.360202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443888dc-db2a-4191-9953-cc3415061e0e admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:42.374216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443888dc-db2a-4191-9953-cc3415061e0e admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:42.379821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adcf01-5244-4b7c-abd2-9efe00e8917d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:42.380475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adcf01-5244-4b7c-abd2-9efe00e8917d admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.380756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adcf01-5244-4b7c-abd2-9efe00e8917d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.381026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76adcf01-5244-4b7c-abd2-9efe00e8917d admin admin] PATH_INFO: `/v3/projects/307abd3eb4bb4b039083f0c0c6f7c55d/users/5429db5f718845eda5929dbee95d8f7a/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.396644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adcf01-5244-4b7c-abd2-9efe00e8917d admin admin] RBAC: Authorizing `identity:create_grant(project_id=307abd3eb4bb4b039083f0c0c6f7c55d, user_id=5429db5f718845eda5929dbee95d8f7a, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:42.399733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76adcf01-5244-4b7c-abd2-9efe00e8917d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:42.414790 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1255/2526] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:42 2022] PUT /identity/v3/projects/307abd3eb4bb4b039083f0c0c6f7c55d/users/5429db5f718845eda5929dbee95d8f7a/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:54:42.427187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e835f0d-c108-485f-8e47-857b15a25a19 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.427476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e835f0d-c108-485f-8e47-857b15a25a19 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.428270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e835f0d-c108-485f-8e47-857b15a25a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.447418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:42.458694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-5e835f0d-c108-485f-8e47-857b15a25a19 None None] MFA Rules not processed for user `5429db5f718845eda5929dbee95d8f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:42.460700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-5e835f0d-c108-485f-8e47-857b15a25a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:42.528937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1272/2527] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:42.534047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adcf01-5244-4b7c-abd2-9efe00e8917d admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:42.573026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:42.599421 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76adcf01-5244-4b7c-abd2-9efe00e8917d admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:42.604082 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766c637-a232-4d77-a5c6-2412430c945c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:42.604689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766c637-a232-4d77-a5c6-2412430c945c admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.604971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766c637-a232-4d77-a5c6-2412430c945c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.605248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766c637-a232-4d77-a5c6-2412430c945c admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.606073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766c637-a232-4d77-a5c6-2412430c945c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:42.608072 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766c637-a232-4d77-a5c6-2412430c945c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:42.636268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1256/2528] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:42 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:42.640364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e835f0d-c108-485f-8e47-857b15a25a19 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:42.653899 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e835f0d-c108-485f-8e47-857b15a25a19 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:42.726836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad035c09-f0d9-4643-a2bc-3f0b424ba6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:42.727445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad035c09-f0d9-4643-a2bc-3f0b424ba6cc admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.727728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad035c09-f0d9-4643-a2bc-3f0b424ba6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.728004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad035c09-f0d9-4643-a2bc-3f0b424ba6cc admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.728635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad035c09-f0d9-4643-a2bc-3f0b424ba6cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:42.730597 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad035c09-f0d9-4643-a2bc-3f0b424ba6cc admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:42.752462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1273/2529] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:42 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:42.756481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766c637-a232-4d77-a5c6-2412430c945c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:42.769170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766c637-a232-4d77-a5c6-2412430c945c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:42.773938 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50007334-8f65-4cf5-a38f-ee591edfd365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:42.774561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50007334-8f65-4cf5-a38f-ee591edfd365 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.774828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50007334-8f65-4cf5-a38f-ee591edfd365 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.775104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50007334-8f65-4cf5-a38f-ee591edfd365 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.775747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50007334-8f65-4cf5-a38f-ee591edfd365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:42.777351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50007334-8f65-4cf5-a38f-ee591edfd365 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:42.790470 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1257/2530] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:42.794670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad035c09-f0d9-4643-a2bc-3f0b424ba6cc admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:42.804683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad035c09-f0d9-4643-a2bc-3f0b424ba6cc admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:42.810096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe1446-7737-4ce0-bf0e-c008c2edb348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:42.810755 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe1446-7737-4ce0-bf0e-c008c2edb348 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.811037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe1446-7737-4ce0-bf0e-c008c2edb348 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.811313 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe1446-7737-4ce0-bf0e-c008c2edb348 admin admin] PATH_INFO: `/v3/projects/5e54db10ab494a5b85dcdca0a0b1bb51/users/86610811bdf54890aa4d6b866dff2a1a/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.826349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe1446-7737-4ce0-bf0e-c008c2edb348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e54db10ab494a5b85dcdca0a0b1bb51, user_id=86610811bdf54890aa4d6b866dff2a1a, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:42.828997 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe1446-7737-4ce0-bf0e-c008c2edb348 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:42.843490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1274/2531] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:42 2022] PUT /identity/v3/projects/5e54db10ab494a5b85dcdca0a0b1bb51/users/86610811bdf54890aa4d6b866dff2a1a/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:42.848117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50007334-8f65-4cf5-a38f-ee591edfd365 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:42.858452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50007334-8f65-4cf5-a38f-ee591edfd365 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:42.922004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949afb1-0982-48c7-a1ce-54709d083b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:42.922619 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949afb1-0982-48c7-a1ce-54709d083b2c admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.922899 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949afb1-0982-48c7-a1ce-54709d083b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.923170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949afb1-0982-48c7-a1ce-54709d083b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.923896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949afb1-0982-48c7-a1ce-54709d083b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:42.925519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949afb1-0982-48c7-a1ce-54709d083b2c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:42.936427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1258/2532] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:54:42.941586 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe1446-7737-4ce0-bf0e-c008c2edb348 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:42.952488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe1446-7737-4ce0-bf0e-c008c2edb348 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:42.959139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb56a6-4fac-4351-a75d-48f958dd777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:42.959919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb56a6-4fac-4351-a75d-48f958dd777b admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.960309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb56a6-4fac-4351-a75d-48f958dd777b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.960681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb56a6-4fac-4351-a75d-48f958dd777b admin admin] PATH_INFO: `/v3/projects/5e54db10ab494a5b85dcdca0a0b1bb51/users/86610811bdf54890aa4d6b866dff2a1a/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:42.962731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb56a6-4fac-4351-a75d-48f958dd777b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e54db10ab494a5b85dcdca0a0b1bb51, user_id=86610811bdf54890aa4d6b866dff2a1a, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:42.967055 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb56a6-4fac-4351-a75d-48f958dd777b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:42.977766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1275/2533] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:42 2022] PUT /identity/v3/projects/5e54db10ab494a5b85dcdca0a0b1bb51/users/86610811bdf54890aa4d6b866dff2a1a/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:54:42.988130 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff62434-2b54-4472-bba6-b13125d26f74 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:42.988414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff62434-2b54-4472-bba6-b13125d26f74 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:42.988696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff62434-2b54-4472-bba6-b13125d26f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:43.018387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-aff62434-2b54-4472-bba6-b13125d26f74 None None] MFA Rules not processed for user `86610811bdf54890aa4d6b866dff2a1a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:43.019659 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-aff62434-2b54-4472-bba6-b13125d26f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:43.109848 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1259/2534] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:43.121000 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb56a6-4fac-4351-a75d-48f958dd777b admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:43.122765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-58eb56a6-4fac-4351-a75d-48f958dd777b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:43.179169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb56a6-4fac-4351-a75d-48f958dd777b admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:43.182726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ac5bd-296f-4c52-8a59-00c5f13e1029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:43.183309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ac5bd-296f-4c52-8a59-00c5f13e1029 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:43.183585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ac5bd-296f-4c52-8a59-00c5f13e1029 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:43.183852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ac5bd-296f-4c52-8a59-00c5f13e1029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:43.190015 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:43.194638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ac5bd-296f-4c52-8a59-00c5f13e1029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:43.196404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ac5bd-296f-4c52-8a59-00c5f13e1029 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:43.207069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1276/2535] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:43.247779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff62434-2b54-4472-bba6-b13125d26f74 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:43.249150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-aff62434-2b54-4472-bba6-b13125d26f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:43.261573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff62434-2b54-4472-bba6-b13125d26f74 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:43.265242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf50f42-3f56-4f07-977a-0880ad6dbf51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:43.265884 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf50f42-3f56-4f07-977a-0880ad6dbf51 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:43.266148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf50f42-3f56-4f07-977a-0880ad6dbf51 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:43.266421 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf50f42-3f56-4f07-977a-0880ad6dbf51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:43.294862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:43.329498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf50f42-3f56-4f07-977a-0880ad6dbf51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:43.331035 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf50f42-3f56-4f07-977a-0880ad6dbf51 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:43.340168 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1260/2536] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:46.437897 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ac5bd-296f-4c52-8a59-00c5f13e1029 service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:46.499110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ac5bd-296f-4c52-8a59-00c5f13e1029 service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:46.504386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:46.505024 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:46.505299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:46.505570 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] PATH_INFO: `/v3/users/3682a93a85624f558ba76630961ca67b` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:46.506777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3682a93a85624f558ba76630961ca67b)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:46.508687 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:46.553795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3682a93a85624f558ba76630961ca67b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:46.554444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3682a93a85624f558ba76630961ca67b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:46.561212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3682a93a85624f558ba76630961ca67b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:46.561869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3682a93a85624f558ba76630961ca67b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:46.574273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3682a93a85624f558ba76630961ca67b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:46.587401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1277/2537] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:46 2022] DELETE /identity/v3/users/3682a93a85624f558ba76630961ca67b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:46.592046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545a574-ca88-445e-9960-3dd66c8f624b None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:46.592508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545a574-ca88-445e-9960-3dd66c8f624b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:46.592629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545a574-ca88-445e-9960-3dd66c8f624b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:46.611916 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-7545a574-ca88-445e-9960-3dd66c8f624b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:46.613108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7545a574-ca88-445e-9960-3dd66c8f624b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:46.670429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1261/2538] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:46.680273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:46.681708 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:46.738367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9806a69-e000-4792-9f8d-44aca1d148d6 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:46.742004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d7cf7-3862-4b6f-920d-682133331e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:46.742601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d7cf7-3862-4b6f-920d-682133331e6f service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:46.742884 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d7cf7-3862-4b6f-920d-682133331e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:46.743184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d7cf7-3862-4b6f-920d-682133331e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:46.752653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d7cf7-3862-4b6f-920d-682133331e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:46.754107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d7cf7-3862-4b6f-920d-682133331e6f service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:46.763338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1278/2539] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:47.125422 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545a574-ca88-445e-9960-3dd66c8f624b None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:47.127677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7545a574-ca88-445e-9960-3dd66c8f624b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:47.143626 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545a574-ca88-445e-9960-3dd66c8f624b None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:47.148708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:47.149359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:47.149641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:47.149951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] PATH_INFO: `/v3/projects/3672584ef1f8479ea0829826e5906999` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:47.151081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3672584ef1f8479ea0829826e5906999)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:47.153037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:47.207367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3672584ef1f8479ea0829826e5906999', 'request_id': 'req-d69d0976-029e-4200-ad7b-a91e341f3a2f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.214261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3672584ef1f8479ea0829826e5906999', 'request_id': 'req-d69d0976-029e-4200-ad7b-a91e341f3a2f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.221409 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3672584ef1f8479ea0829826e5906999', 'request_id': 'req-d69d0976-029e-4200-ad7b-a91e341f3a2f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.236767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] The token cache is being invalidate because project 3672584ef1f8479ea0829826e5906999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:47.238565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.238565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.242696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1262/2540] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:47 2022] DELETE /identity/v3/projects/3672584ef1f8479ea0829826e5906999 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:47.244006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d7cf7-3862-4b6f-920d-682133331e6f service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:47.255974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d7cf7-3862-4b6f-920d-682133331e6f service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:47.285651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:47.333907 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:47.334478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:47.334758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:47.335033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] PATH_INFO: `/v3/users/5429db5f718845eda5929dbee95d8f7a` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:47.336269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5429db5f718845eda5929dbee95d8f7a)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:47.338670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:47.395106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5429db5f718845eda5929dbee95d8f7a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.395793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5429db5f718845eda5929dbee95d8f7a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.403476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5429db5f718845eda5929dbee95d8f7a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.404223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5429db5f718845eda5929dbee95d8f7a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.418966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5429db5f718845eda5929dbee95d8f7a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.438274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1279/2541] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:47 2022] DELETE /identity/v3/users/5429db5f718845eda5929dbee95d8f7a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:54:47.771696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:47.773506 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:47.809342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d0976-029e-4200-ad7b-a91e341f3a2f admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:47.864183 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:47.878185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:47.878867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:47.879204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:47.879499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] PATH_INFO: `/v3/projects/307abd3eb4bb4b039083f0c0c6f7c55d` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:47.881090 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=307abd3eb4bb4b039083f0c0c6f7c55d)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:47.883045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:47.934436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '307abd3eb4bb4b039083f0c0c6f7c55d', 'request_id': 'req-81531fbb-8ef0-4ce9-b16f-0a3219f59968'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.941283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '307abd3eb4bb4b039083f0c0c6f7c55d', 'request_id': 'req-81531fbb-8ef0-4ce9-b16f-0a3219f59968'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.947740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '307abd3eb4bb4b039083f0c0c6f7c55d', 'request_id': 'req-81531fbb-8ef0-4ce9-b16f-0a3219f59968'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.960236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] The token cache is being invalidate because project 307abd3eb4bb4b039083f0c0c6f7c55d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:47.960528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.961175 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:47.962798 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1263/2542] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:47 2022] DELETE /identity/v3/projects/307abd3eb4bb4b039083f0c0c6f7c55d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:47.966878 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:47.968503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:47.982299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40e05e-6795-4649-9d0e-967fc552a824 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:48.038978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f11f1-77a1-40e2-acdc-8d4fce506441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:48.039758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f11f1-77a1-40e2-acdc-8d4fce506441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:48.040040 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f11f1-77a1-40e2-acdc-8d4fce506441 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:48.040314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f11f1-77a1-40e2-acdc-8d4fce506441 admin admin] PATH_INFO: `/v3/users/86610811bdf54890aa4d6b866dff2a1a` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:48.041466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f11f1-77a1-40e2-acdc-8d4fce506441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86610811bdf54890aa4d6b866dff2a1a)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:48.043442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f11f1-77a1-40e2-acdc-8d4fce506441 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:48.084276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:48.094462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-344f11f1-77a1-40e2-acdc-8d4fce506441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86610811bdf54890aa4d6b866dff2a1a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:48.096289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-344f11f1-77a1-40e2-acdc-8d4fce506441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86610811bdf54890aa4d6b866dff2a1a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:48.105534 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-344f11f1-77a1-40e2-acdc-8d4fce506441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86610811bdf54890aa4d6b866dff2a1a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:48.106338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-344f11f1-77a1-40e2-acdc-8d4fce506441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86610811bdf54890aa4d6b866dff2a1a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:48.122281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-344f11f1-77a1-40e2-acdc-8d4fce506441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86610811bdf54890aa4d6b866dff2a1a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:48.144225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1280/2543] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:47 2022] DELETE /identity/v3/users/86610811bdf54890aa4d6b866dff2a1a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:48.464076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:48.465493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:48.483900 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81531fbb-8ef0-4ce9-b16f-0a3219f59968 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:48.540672 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dde7e1-a379-48ca-82cd-e7d3c5424d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:48.541304 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dde7e1-a379-48ca-82cd-e7d3c5424d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:48.541583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dde7e1-a379-48ca-82cd-e7d3c5424d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:48.541891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dde7e1-a379-48ca-82cd-e7d3c5424d46 admin admin] PATH_INFO: `/v3/projects/5e54db10ab494a5b85dcdca0a0b1bb51` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:48.543019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dde7e1-a379-48ca-82cd-e7d3c5424d46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e54db10ab494a5b85dcdca0a0b1bb51)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:48.545073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dde7e1-a379-48ca-82cd-e7d3c5424d46 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:48.592061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-16dde7e1-a379-48ca-82cd-e7d3c5424d46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e54db10ab494a5b85dcdca0a0b1bb51', 'request_id': 'req-16dde7e1-a379-48ca-82cd-e7d3c5424d46'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:48.598290 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-16dde7e1-a379-48ca-82cd-e7d3c5424d46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e54db10ab494a5b85dcdca0a0b1bb51', 'request_id': 'req-16dde7e1-a379-48ca-82cd-e7d3c5424d46'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:48.604239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-16dde7e1-a379-48ca-82cd-e7d3c5424d46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e54db10ab494a5b85dcdca0a0b1bb51', 'request_id': 'req-16dde7e1-a379-48ca-82cd-e7d3c5424d46'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:48.616973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-16dde7e1-a379-48ca-82cd-e7d3c5424d46 admin admin] The token cache is being invalidate because project 5e54db10ab494a5b85dcdca0a0b1bb51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:48.617265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-16dde7e1-a379-48ca-82cd-e7d3c5424d46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:48.617927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-16dde7e1-a379-48ca-82cd-e7d3c5424d46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:48.619424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1264/2544] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:48 2022] DELETE /identity/v3/projects/5e54db10ab494a5b85dcdca0a0b1bb51 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:48.625308 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6470a6e-0c8b-40c2-a211-d5725d037ace None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:48.625818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6470a6e-0c8b-40c2-a211-d5725d037ace None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:48.625920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6470a6e-0c8b-40c2-a211-d5725d037ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:48.645938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-e6470a6e-0c8b-40c2-a211-d5725d037ace None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:48.647151 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e6470a6e-0c8b-40c2-a211-d5725d037ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:48.711341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1281/2545] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:48.721362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab44710-b8df-4124-bc5b-b229a6114c9e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:48.724757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab44710-b8df-4124-bc5b-b229a6114c9e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:48.724757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab44710-b8df-4124-bc5b-b229a6114c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:48.747150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-bab44710-b8df-4124-bc5b-b229a6114c9e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:48.749065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-bab44710-b8df-4124-bc5b-b229a6114c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:48.759230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1265/2546] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:54:48.763370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6470a6e-0c8b-40c2-a211-d5725d037ace None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:48.775165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6470a6e-0c8b-40c2-a211-d5725d037ace None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:48.780481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82237aca-89fe-4897-8b64-b5eca73568e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:48.781525 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82237aca-89fe-4897-8b64-b5eca73568e5 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:48.781525 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82237aca-89fe-4897-8b64-b5eca73568e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:48.781818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82237aca-89fe-4897-8b64-b5eca73568e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:48.782329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82237aca-89fe-4897-8b64-b5eca73568e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:54:48.782670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82237aca-89fe-4897-8b64-b5eca73568e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:48.784368 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82237aca-89fe-4897-8b64-b5eca73568e5 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:48.799432 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1282/2547] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:54:48.811368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab44710-b8df-4124-bc5b-b229a6114c9e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:48.826501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab44710-b8df-4124-bc5b-b229a6114c9e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:48.831374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3c630-ff16-4ee6-8681-567332a91534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:48.832115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3c630-ff16-4ee6-8681-567332a91534 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:48.832400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3c630-ff16-4ee6-8681-567332a91534 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:48.832677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3c630-ff16-4ee6-8681-567332a91534 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:48.833585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3c630-ff16-4ee6-8681-567332a91534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:48.835539 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3c630-ff16-4ee6-8681-567332a91534 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:48.840416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:48.874122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1266/2548] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:48 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:48.878251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82237aca-89fe-4897-8b64-b5eca73568e5 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:48.890167 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82237aca-89fe-4897-8b64-b5eca73568e5 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:48.950278 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77815b-19e3-421d-9758-c3ae7d02e9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:48.950804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77815b-19e3-421d-9758-c3ae7d02e9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:48.951082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77815b-19e3-421d-9758-c3ae7d02e9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:48.951353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77815b-19e3-421d-9758-c3ae7d02e9f8 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:48.951999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77815b-19e3-421d-9758-c3ae7d02e9f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:48.953965 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77815b-19e3-421d-9758-c3ae7d02e9f8 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:48.977136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1283/2549] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:48 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:54:48.981081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3c630-ff16-4ee6-8681-567332a91534 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:48.992389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3c630-ff16-4ee6-8681-567332a91534 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:48.997180 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238edbf5-7347-4717-812f-f416647fa031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:48.998431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238edbf5-7347-4717-812f-f416647fa031 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:48.998724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238edbf5-7347-4717-812f-f416647fa031 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:48.998980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238edbf5-7347-4717-812f-f416647fa031 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:48.999864 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238edbf5-7347-4717-812f-f416647fa031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:49.001597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238edbf5-7347-4717-812f-f416647fa031 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:49.012919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1267/2550] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:54:49.016400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77815b-19e3-421d-9758-c3ae7d02e9f8 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:49.026775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77815b-19e3-421d-9758-c3ae7d02e9f8 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:49.031487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e76dfb-b23c-4cb5-9367-d2419af4cd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:49.032091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e76dfb-b23c-4cb5-9367-d2419af4cd13 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:49.032867 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e76dfb-b23c-4cb5-9367-d2419af4cd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:49.033149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e76dfb-b23c-4cb5-9367-d2419af4cd13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:49.033837 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e76dfb-b23c-4cb5-9367-d2419af4cd13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:49.035458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e76dfb-b23c-4cb5-9367-d2419af4cd13 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:49.047621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1284/2551] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:54:49.052447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238edbf5-7347-4717-812f-f416647fa031 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:49.065451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238edbf5-7347-4717-812f-f416647fa031 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:49.070123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f9412-c558-4ae2-a1f8-d54fa662b3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:49.070825 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f9412-c558-4ae2-a1f8-d54fa662b3b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:49.071107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f9412-c558-4ae2-a1f8-d54fa662b3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:49.071377 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f9412-c558-4ae2-a1f8-d54fa662b3b1 admin admin] PATH_INFO: `/v3/projects/ae62391af5b44142a2f8584172a36e67/users/6921d0b4d4f64990ab053cd130460af8/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:49.089780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f9412-c558-4ae2-a1f8-d54fa662b3b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae62391af5b44142a2f8584172a36e67, user_id=6921d0b4d4f64990ab053cd130460af8, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:49.092464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f9412-c558-4ae2-a1f8-d54fa662b3b1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:49.108260 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1268/2552] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:49 2022] PUT /identity/v3/projects/ae62391af5b44142a2f8584172a36e67/users/6921d0b4d4f64990ab053cd130460af8/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:54:49.121038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026914be-3c9a-4564-8436-3c49d6962898 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:49.121826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026914be-3c9a-4564-8436-3c49d6962898 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:49.122230 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026914be-3c9a-4564-8436-3c49d6962898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:49.159023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-026914be-3c9a-4564-8436-3c49d6962898 None None] MFA Rules not processed for user `6921d0b4d4f64990ab053cd130460af8`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:49.160563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-026914be-3c9a-4564-8436-3c49d6962898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:49.173917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:49.242582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1285/2553] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:49.255256 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f9412-c558-4ae2-a1f8-d54fa662b3b1 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:49.258238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a62f9412-c558-4ae2-a1f8-d54fa662b3b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:49.339332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f9412-c558-4ae2-a1f8-d54fa662b3b1 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:49.343371 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a38a84-9364-4790-bfbb-8261f6c49cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:49.343980 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a38a84-9364-4790-bfbb-8261f6c49cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:49.344259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a38a84-9364-4790-bfbb-8261f6c49cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:49.344527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a38a84-9364-4790-bfbb-8261f6c49cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:49.354465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a38a84-9364-4790-bfbb-8261f6c49cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:49.355917 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a38a84-9364-4790-bfbb-8261f6c49cc2 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:49.382295 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1269/2554] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:49.411303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026914be-3c9a-4564-8436-3c49d6962898 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:49.509448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026914be-3c9a-4564-8436-3c49d6962898 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:49.525811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:49.526594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:49.526995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:49.528310 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] PATH_INFO: `/v3/users/6921d0b4d4f64990ab053cd130460af8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:49.530439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6921d0b4d4f64990ab053cd130460af8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:49.533383 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:49.616193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6921d0b4d4f64990ab053cd130460af8'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:49.616866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6921d0b4d4f64990ab053cd130460af8'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:49.623836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6921d0b4d4f64990ab053cd130460af8'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:49.624509 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6921d0b4d4f64990ab053cd130460af8'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:49.638109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6921d0b4d4f64990ab053cd130460af8'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:49.651926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1286/2555] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:49 2022] DELETE /identity/v3/users/6921d0b4d4f64990ab053cd130460af8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:49.656456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d342a-3319-46ff-8cd5-35d519823f2b None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:49.656961 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d342a-3319-46ff-8cd5-35d519823f2b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:49.658256 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d342a-3319-46ff-8cd5-35d519823f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:49.678868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-4f4d342a-3319-46ff-8cd5-35d519823f2b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:49.680058 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4f4d342a-3319-46ff-8cd5-35d519823f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:49.741569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1270/2556] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:49.751674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:49.753191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:49.786119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:49.816474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b358e16-cd41-41af-b8eb-5944f198db1e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:49.824286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8583fbf-d7cb-4ba8-8cf1-67de1f5b2027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:49.824895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8583fbf-d7cb-4ba8-8cf1-67de1f5b2027 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:49.825201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8583fbf-d7cb-4ba8-8cf1-67de1f5b2027 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:49.825471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8583fbf-d7cb-4ba8-8cf1-67de1f5b2027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:49.835826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8583fbf-d7cb-4ba8-8cf1-67de1f5b2027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:49.837802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8583fbf-d7cb-4ba8-8cf1-67de1f5b2027 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:49.847744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1287/2557] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:50.408219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d342a-3319-46ff-8cd5-35d519823f2b None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:50.409813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4f4d342a-3319-46ff-8cd5-35d519823f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:50.421560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d342a-3319-46ff-8cd5-35d519823f2b None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:50.426846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7afbfe5-f94e-481b-b885-fada963df63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:50.427478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afbfe5-f94e-481b-b885-fada963df63f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:50.427753 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afbfe5-f94e-481b-b885-fada963df63f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:50.428044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afbfe5-f94e-481b-b885-fada963df63f admin admin] PATH_INFO: `/v3/projects/ae62391af5b44142a2f8584172a36e67` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:50.429189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afbfe5-f94e-481b-b885-fada963df63f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae62391af5b44142a2f8584172a36e67)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:50.431085 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7afbfe5-f94e-481b-b885-fada963df63f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:50.481188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b7afbfe5-f94e-481b-b885-fada963df63f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae62391af5b44142a2f8584172a36e67', 'request_id': 'req-b7afbfe5-f94e-481b-b885-fada963df63f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:50.482482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:50.489517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b7afbfe5-f94e-481b-b885-fada963df63f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae62391af5b44142a2f8584172a36e67', 'request_id': 'req-b7afbfe5-f94e-481b-b885-fada963df63f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:50.498216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b7afbfe5-f94e-481b-b885-fada963df63f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae62391af5b44142a2f8584172a36e67', 'request_id': 'req-b7afbfe5-f94e-481b-b885-fada963df63f'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:50.511575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b7afbfe5-f94e-481b-b885-fada963df63f admin admin] The token cache is being invalidate because project ae62391af5b44142a2f8584172a36e67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:50.511870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b7afbfe5-f94e-481b-b885-fada963df63f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:50.512496 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b7afbfe5-f94e-481b-b885-fada963df63f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:50.514007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1271/2558] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:50 2022] DELETE /identity/v3/projects/ae62391af5b44142a2f8584172a36e67 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:50.518801 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aff0c-6762-4521-9e6c-857704e7424f None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:50.519109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aff0c-6762-4521-9e6c-857704e7424f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:50.519396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aff0c-6762-4521-9e6c-857704e7424f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:50.540007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-558aff0c-6762-4521-9e6c-857704e7424f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:50.541238 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-558aff0c-6762-4521-9e6c-857704e7424f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:50.618097 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1288/2559] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:50.620430 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ff8bb-7c20-48ea-930b-f129965139d8 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:50.620896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ff8bb-7c20-48ea-930b-f129965139d8 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:50.621272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ff8bb-7c20-48ea-930b-f129965139d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:50.644146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-562ff8bb-7c20-48ea-930b-f129965139d8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:50.645614 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-562ff8bb-7c20-48ea-930b-f129965139d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:50.654414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1272/2560] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:50.663862 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558aff0c-6762-4521-9e6c-857704e7424f None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:50.679062 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558aff0c-6762-4521-9e6c-857704e7424f None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:50.685867 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fe2b8-8cc0-4c68-8f46-9e4dbbb01a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:50.686702 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fe2b8-8cc0-4c68-8f46-9e4dbbb01a33 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:50.687177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fe2b8-8cc0-4c68-8f46-9e4dbbb01a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:50.688146 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6fe2b8-8cc0-4c68-8f46-9e4dbbb01a33 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:50.689393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fe2b8-8cc0-4c68-8f46-9e4dbbb01a33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:54:50.689895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fe2b8-8cc0-4c68-8f46-9e4dbbb01a33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:50.692296 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6fe2b8-8cc0-4c68-8f46-9e4dbbb01a33 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:50.708245 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1289/2561] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:54:50.712208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ff8bb-7c20-48ea-930b-f129965139d8 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:50.724275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ff8bb-7c20-48ea-930b-f129965139d8 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:50.729290 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419cfa5-0383-4049-b9e9-09dbd0757713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:50.729945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419cfa5-0383-4049-b9e9-09dbd0757713 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:50.730732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419cfa5-0383-4049-b9e9-09dbd0757713 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:50.730732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419cfa5-0383-4049-b9e9-09dbd0757713 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:50.731121 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419cfa5-0383-4049-b9e9-09dbd0757713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:50.733033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419cfa5-0383-4049-b9e9-09dbd0757713 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:50.765743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1273/2562] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:54:50 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:54:50.768374 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fe2b8-8cc0-4c68-8f46-9e4dbbb01a33 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:50.779215 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6fe2b8-8cc0-4c68-8f46-9e4dbbb01a33 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:50.840092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61df0bc-7cd7-4da2-a77e-86f7d4492ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:50.840887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61df0bc-7cd7-4da2-a77e-86f7d4492ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:50.841352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61df0bc-7cd7-4da2-a77e-86f7d4492ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:50.841841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61df0bc-7cd7-4da2-a77e-86f7d4492ad6 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:50.842666 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61df0bc-7cd7-4da2-a77e-86f7d4492ad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:50.844711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61df0bc-7cd7-4da2-a77e-86f7d4492ad6 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:50.876061 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1290/2563] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:54:50 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:54:50.877198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419cfa5-0383-4049-b9e9-09dbd0757713 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:50.888030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419cfa5-0383-4049-b9e9-09dbd0757713 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:50.893434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f92d2-31fa-4880-a2a9-37b8ddd69dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:50.894077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f92d2-31fa-4880-a2a9-37b8ddd69dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:50.894360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f92d2-31fa-4880-a2a9-37b8ddd69dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:50.894783 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f92d2-31fa-4880-a2a9-37b8ddd69dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:50.895428 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f92d2-31fa-4880-a2a9-37b8ddd69dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:50.897030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f92d2-31fa-4880-a2a9-37b8ddd69dd2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:50.908452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1274/2564] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:54:50.912558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61df0bc-7cd7-4da2-a77e-86f7d4492ad6 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:50.922649 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61df0bc-7cd7-4da2-a77e-86f7d4492ad6 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:50.927345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46702876-1568-4866-8a2d-aa50dbdb5fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:50.927943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46702876-1568-4866-8a2d-aa50dbdb5fbf admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:50.928224 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46702876-1568-4866-8a2d-aa50dbdb5fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:50.928500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46702876-1568-4866-8a2d-aa50dbdb5fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:50.929169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46702876-1568-4866-8a2d-aa50dbdb5fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:50.930808 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46702876-1568-4866-8a2d-aa50dbdb5fbf admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:50.941514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1291/2565] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:54:50.945882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f92d2-31fa-4880-a2a9-37b8ddd69dd2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:50.956359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f92d2-31fa-4880-a2a9-37b8ddd69dd2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:50.961511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb599d05-91d2-467f-9c74-451e2dfc8a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:50.962287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb599d05-91d2-467f-9c74-451e2dfc8a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:50.962634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb599d05-91d2-467f-9c74-451e2dfc8a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:50.962978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb599d05-91d2-467f-9c74-451e2dfc8a61 admin admin] PATH_INFO: `/v3/projects/6cdd2981a5ec41e4be71ca74dd8de32e/users/519098a662ad4695be9e6475d3f85792/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:50.981123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb599d05-91d2-467f-9c74-451e2dfc8a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cdd2981a5ec41e4be71ca74dd8de32e, user_id=519098a662ad4695be9e6475d3f85792, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:50.983999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb599d05-91d2-467f-9c74-451e2dfc8a61 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:50.999472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1275/2566] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:54:50 2022] PUT /identity/v3/projects/6cdd2981a5ec41e4be71ca74dd8de32e/users/519098a662ad4695be9e6475d3f85792/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:54:51.010753 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512429e1-a8f5-44a6-82c9-6fd6d8ecceaf None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:51.011066 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512429e1-a8f5-44a6-82c9-6fd6d8ecceaf None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:51.011356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512429e1-a8f5-44a6-82c9-6fd6d8ecceaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:51.044776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-512429e1-a8f5-44a6-82c9-6fd6d8ecceaf None None] MFA Rules not processed for user `519098a662ad4695be9e6475d3f85792`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:51.046249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-512429e1-a8f5-44a6-82c9-6fd6d8ecceaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:51.075023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:51.136911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1292/2567] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:51.151836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb599d05-91d2-467f-9c74-451e2dfc8a61 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:51.154120 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-cb599d05-91d2-467f-9c74-451e2dfc8a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:51.236927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb599d05-91d2-467f-9c74-451e2dfc8a61 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:51.240917 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fada80e-c2cf-47df-b382-2564bb49ab28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:51.241534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada80e-c2cf-47df-b382-2564bb49ab28 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:51.241853 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada80e-c2cf-47df-b382-2564bb49ab28 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:51.242134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fada80e-c2cf-47df-b382-2564bb49ab28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:51.252473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fada80e-c2cf-47df-b382-2564bb49ab28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:51.254038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fada80e-c2cf-47df-b382-2564bb49ab28 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:51.264556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1276/2568] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:51.291456 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512429e1-a8f5-44a6-82c9-6fd6d8ecceaf None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:51.356970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512429e1-a8f5-44a6-82c9-6fd6d8ecceaf None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:51.361608 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c0411-817a-459a-b932-43103117294d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:51.362265 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c0411-817a-459a-b932-43103117294d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:51.362543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c0411-817a-459a-b932-43103117294d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:51.362815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c0411-817a-459a-b932-43103117294d admin admin] PATH_INFO: `/v3/users/519098a662ad4695be9e6475d3f85792` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:51.363989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c0411-817a-459a-b932-43103117294d admin admin] RBAC: Authorizing `identity:delete_user(user_id=519098a662ad4695be9e6475d3f85792)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:51.365950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133c0411-817a-459a-b932-43103117294d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:51.417068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-133c0411-817a-459a-b932-43103117294d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519098a662ad4695be9e6475d3f85792'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:51.417757 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-133c0411-817a-459a-b932-43103117294d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519098a662ad4695be9e6475d3f85792'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:51.424749 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-133c0411-817a-459a-b932-43103117294d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519098a662ad4695be9e6475d3f85792'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:51.425402 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-133c0411-817a-459a-b932-43103117294d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519098a662ad4695be9e6475d3f85792'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:51.437871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-133c0411-817a-459a-b932-43103117294d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519098a662ad4695be9e6475d3f85792'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:51.452700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1293/2569] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:54:51 2022] DELETE /identity/v3/users/519098a662ad4695be9e6475d3f85792 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:54:51.458327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad2f54-71bb-4413-9dd9-1a5f8b028470 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:51.458820 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad2f54-71bb-4413-9dd9-1a5f8b028470 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:51.458943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad2f54-71bb-4413-9dd9-1a5f8b028470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:51.478009 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d5ad2f54-71bb-4413-9dd9-1a5f8b028470 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:54:51.479202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d5ad2f54-71bb-4413-9dd9-1a5f8b028470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:51.538765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1277/2570] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:54:51.549499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c0411-817a-459a-b932-43103117294d admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:51.551012 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-133c0411-817a-459a-b932-43103117294d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:51.604363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:51.611299 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c0411-817a-459a-b932-43103117294d admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:51.616834 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bb317-c9f1-49dd-89b5-f2655c7623c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:51.617475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bb317-c9f1-49dd-89b5-f2655c7623c5 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:51.617781 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bb317-c9f1-49dd-89b5-f2655c7623c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:51.618054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bb317-c9f1-49dd-89b5-f2655c7623c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:51.628519 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bb317-c9f1-49dd-89b5-f2655c7623c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:51.629981 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bb317-c9f1-49dd-89b5-f2655c7623c5 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:51.639963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1294/2571] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:54:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:54:52.002171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad2f54-71bb-4413-9dd9-1a5f8b028470 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:54:52.003706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d5ad2f54-71bb-4413-9dd9-1a5f8b028470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:54:52.017406 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad2f54-71bb-4413-9dd9-1a5f8b028470 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:54:52.030071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:54:52.031977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:54:52.032846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:54:52.041279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] PATH_INFO: `/v3/projects/6cdd2981a5ec41e4be71ca74dd8de32e` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:54:52.042775 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cdd2981a5ec41e4be71ca74dd8de32e)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:54:52.045163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:54:52.056842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:54:52.096580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cdd2981a5ec41e4be71ca74dd8de32e', 'request_id': 'req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:52.104163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cdd2981a5ec41e4be71ca74dd8de32e', 'request_id': 'req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:52.111327 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cdd2981a5ec41e4be71ca74dd8de32e', 'request_id': 'req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:52.125956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] The token cache is being invalidate because project 6cdd2981a5ec41e4be71ca74dd8de32e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:54:52.126500 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:52.127343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:54:52.128879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1278/2572] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:54:52 2022] DELETE /identity/v3/projects/6cdd2981a5ec41e4be71ca74dd8de32e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:55:01.635578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=5 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:02.121565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:08.199076 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a1db3-0aff-458a-9768-7153d0d352f7 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:08.199586 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a1db3-0aff-458a-9768-7153d0d352f7 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:08.199716 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a1db3-0aff-458a-9768-7153d0d352f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:08.230618 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d51a1db3-0aff-458a-9768-7153d0d352f7 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:08.231921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d51a1db3-0aff-458a-9768-7153d0d352f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:08.292663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1295/2573] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:55:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:55:08.339664 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:08.341188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:08.411148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc5ac4-d155-4ce1-b3be-63caf8c311f7 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:08.416224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21ea4c-d18a-4e87-9fae-dd8edaa8dea9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:08.416819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21ea4c-d18a-4e87-9fae-dd8edaa8dea9 service swift] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:08.417095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21ea4c-d18a-4e87-9fae-dd8edaa8dea9 service swift] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:08.417360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21ea4c-d18a-4e87-9fae-dd8edaa8dea9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:08.425155 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21ea4c-d18a-4e87-9fae-dd8edaa8dea9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:08.426751 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21ea4c-d18a-4e87-9fae-dd8edaa8dea9 service swift] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:08.434532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1279/2574] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:55:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:55:18.285525 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=32 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:18.431362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=32 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:24.140342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67d744-fc4d-4d81-9b79-440611e99d89 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:24.140895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67d744-fc4d-4d81-9b79-440611e99d89 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:24.140895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67d744-fc4d-4d81-9b79-440611e99d89 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:24.142259 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1296/2575] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:55:24.151013 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e6562-d331-4690-9e2d-fb0e5496a02a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:24.151508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e6562-d331-4690-9e2d-fb0e5496a02a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:24.151689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e6562-d331-4690-9e2d-fb0e5496a02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:24.182922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-023e6562-d331-4690-9e2d-fb0e5496a02a None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:24.184147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-023e6562-d331-4690-9e2d-fb0e5496a02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:24.263797 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1280/2576] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:55:24.276636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67d744-fc4d-4d81-9b79-440611e99d89 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:24.278380 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4a67d744-fc4d-4d81-9b79-440611e99d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:24.335951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67d744-fc4d-4d81-9b79-440611e99d89 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:24.339469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666603b-60f6-4304-bb6a-7e23b60ed0fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '913a866ddc074b7f9aa142c70da98a69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:24.340062 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666603b-60f6-4304-bb6a-7e23b60ed0fb service glance] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:24.340338 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666603b-60f6-4304-bb6a-7e23b60ed0fb service glance] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:24.340604 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666603b-60f6-4304-bb6a-7e23b60ed0fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:24.350445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666603b-60f6-4304-bb6a-7e23b60ed0fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:24.351863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666603b-60f6-4304-bb6a-7e23b60ed0fb service glance] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:24.362436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1297/2577] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:55:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:55:24.430589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113cb34-6c19-4de7-b0eb-e37d7361687d None None] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:24.430869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113cb34-6c19-4de7-b0eb-e37d7361687d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:24.431185 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113cb34-6c19-4de7-b0eb-e37d7361687d None None] PATH_INFO: `/` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:24.432562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1281/2578] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:55:24.437432 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea92c9-8984-4cbd-aed2-3d615e493da7 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:24.437656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea92c9-8984-4cbd-aed2-3d615e493da7 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:24.437986 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea92c9-8984-4cbd-aed2-3d615e493da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:24.457993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-1aea92c9-8984-4cbd-aed2-3d615e493da7 None None] MFA Rules not processed for user `50d3b75cbc5c4d1686f4d93248e04746`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:24.459185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1aea92c9-8984-4cbd-aed2-3d615e493da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:24.508661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1298/2579] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:55:24.520796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113cb34-6c19-4de7-b0eb-e37d7361687d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:24.522518 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b113cb34-6c19-4de7-b0eb-e37d7361687d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:24.535903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113cb34-6c19-4de7-b0eb-e37d7361687d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:24.539409 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ec9f5-4de4-4db1-a420-8980da80406f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:24.539989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ec9f5-4de4-4db1-a420-8980da80406f service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:24.540267 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ec9f5-4de4-4db1-a420-8980da80406f service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:24.540536 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ec9f5-4de4-4db1-a420-8980da80406f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:24.547423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ec9f5-4de4-4db1-a420-8980da80406f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:24.548904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ec9f5-4de4-4db1-a420-8980da80406f service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:24.556939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1282/2580] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:55:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:55:24.640852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dda41d-2a27-4161-90d1-f302d80beed3 None None] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:24.641136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dda41d-2a27-4161-90d1-f302d80beed3 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:24.641407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dda41d-2a27-4161-90d1-f302d80beed3 None None] PATH_INFO: `/` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:24.643293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1299/2581] 10.222.0.26 () {54 vars in 921 bytes} [Tue Oct 4 08:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 08:55:24.648416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fc0db-aed7-4702-b44c-da38d7d00e65 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:24.648732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fc0db-aed7-4702-b44c-da38d7d00e65 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:24.649005 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fc0db-aed7-4702-b44c-da38d7d00e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:24.669202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-e96fc0db-aed7-4702-b44c-da38d7d00e65 None None] MFA Rules not processed for user `ee8a19f4d9704408a8b737095e5885f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:24.670439 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e96fc0db-aed7-4702-b44c-da38d7d00e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:24.679695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1283/2582] 10.222.0.26 () {58 vars in 1018 bytes} [Tue Oct 4 08:55:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:55:24.690319 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dda41d-2a27-4161-90d1-f302d80beed3 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:24.700968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dda41d-2a27-4161-90d1-f302d80beed3 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:24.704492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495a91a-ca45-45f7-a7e2-232cc19ac34d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:24.705075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495a91a-ca45-45f7-a7e2-232cc19ac34d service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:24.705944 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495a91a-ca45-45f7-a7e2-232cc19ac34d service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:24.706169 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495a91a-ca45-45f7-a7e2-232cc19ac34d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:24.716135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495a91a-ca45-45f7-a7e2-232cc19ac34d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:24.717550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495a91a-ca45-45f7-a7e2-232cc19ac34d service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:24.728406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1300/2583] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:55:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:55:26.620902 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a0922-e1d4-4be5-ab36-888870dc14d5 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:26.621445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a0922-e1d4-4be5-ab36-888870dc14d5 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:26.621530 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a0922-e1d4-4be5-ab36-888870dc14d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:26.640813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-b11a0922-e1d4-4be5-ab36-888870dc14d5 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:26.642087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b11a0922-e1d4-4be5-ab36-888870dc14d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:26.655288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1284/2584] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 08:55:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:55:26.668972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495a91a-ca45-45f7-a7e2-232cc19ac34d service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:26.682176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495a91a-ca45-45f7-a7e2-232cc19ac34d service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:26.687291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966a916-a448-462a-b7fb-5f5cd93b3aa8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:26.688048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966a916-a448-462a-b7fb-5f5cd93b3aa8 service swift] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:26.688394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966a916-a448-462a-b7fb-5f5cd93b3aa8 service swift] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:26.688737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966a916-a448-462a-b7fb-5f5cd93b3aa8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:26.697402 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966a916-a448-462a-b7fb-5f5cd93b3aa8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:26.699072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966a916-a448-462a-b7fb-5f5cd93b3aa8 service swift] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:26.709158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1301/2585] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:55:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:55:36.635213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=52 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:36.704301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=61 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:38.558389 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a0922-e1d4-4be5-ab36-888870dc14d5 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:38.559871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b11a0922-e1d4-4be5-ab36-888870dc14d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:38.571079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a0922-e1d4-4be5-ab36-888870dc14d5 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:38.574650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21264fd6-db37-4da9-88cb-8d76a4e3737e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:38.575241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21264fd6-db37-4da9-88cb-8d76a4e3737e service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:38.575521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21264fd6-db37-4da9-88cb-8d76a4e3737e service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:38.575789 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21264fd6-db37-4da9-88cb-8d76a4e3737e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:38.577425 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-21264fd6-db37-4da9-88cb-8d76a4e3737e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:38.590880 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21264fd6-db37-4da9-88cb-8d76a4e3737e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:38.592349 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21264fd6-db37-4da9-88cb-8d76a4e3737e service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:38.604248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1285/2586] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:55:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:55:39.788298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966a916-a448-462a-b7fb-5f5cd93b3aa8 service swift] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:39.797495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966a916-a448-462a-b7fb-5f5cd93b3aa8 service swift] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:39.801593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5550f48-8d6a-4519-a722-7d063ade4d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:39.802203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5550f48-8d6a-4519-a722-7d063ade4d85 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:39.802481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5550f48-8d6a-4519-a722-7d063ade4d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:39.802752 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5550f48-8d6a-4519-a722-7d063ade4d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:39.804444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c5550f48-8d6a-4519-a722-7d063ade4d85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:39.817163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5550f48-8d6a-4519-a722-7d063ade4d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:39.818610 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5550f48-8d6a-4519-a722-7d063ade4d85 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:39.829026 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1302/2587] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:55:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:55:48.599045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=6 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:49.824434 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=6 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:55:50.268451 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da250a8c-d3b7-4e29-8dcb-3cbc35d659e9 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:50.268927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da250a8c-d3b7-4e29-8dcb-3cbc35d659e9 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:50.269032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da250a8c-d3b7-4e29-8dcb-3cbc35d659e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:50.288236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-da250a8c-d3b7-4e29-8dcb-3cbc35d659e9 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:55:50.289442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-da250a8c-d3b7-4e29-8dcb-3cbc35d659e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:55:50.296657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1286/2588] 10.222.0.26 () {58 vars in 967 bytes} [Tue Oct 4 08:55:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:55:50.308968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5550f48-8d6a-4519-a722-7d063ade4d85 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:55:50.320516 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5550f48-8d6a-4519-a722-7d063ade4d85 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:55:50.325453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c43ea-900a-469b-a23e-0ea807eeda63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:55:50.326072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c43ea-900a-469b-a23e-0ea807eeda63 service swift] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:55:50.326355 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c43ea-900a-469b-a23e-0ea807eeda63 service swift] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:55:50.326628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c43ea-900a-469b-a23e-0ea807eeda63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:55:50.344690 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c43ea-900a-469b-a23e-0ea807eeda63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:55:50.346361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c43ea-900a-469b-a23e-0ea807eeda63 service swift] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:55:50.366195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1303/2589] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 08:55:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 08:56:00.283129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:00.357129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=6 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:15.721853 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da250a8c-d3b7-4e29-8dcb-3cbc35d659e9 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:15.723247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-da250a8c-d3b7-4e29-8dcb-3cbc35d659e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:15.779181 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da250a8c-d3b7-4e29-8dcb-3cbc35d659e9 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:15.783286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f338e-89b5-4558-9697-42d5fa410d2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:15.783879 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f338e-89b5-4558-9697-42d5fa410d2c service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:15.784160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f338e-89b5-4558-9697-42d5fa410d2c service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:15.784433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f338e-89b5-4558-9697-42d5fa410d2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:15.786140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e53f338e-89b5-4558-9697-42d5fa410d2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:15.842541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f338e-89b5-4558-9697-42d5fa410d2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:15.844549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f338e-89b5-4558-9697-42d5fa410d2c service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:15.853521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1287/2590] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:25.849411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=48 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:27.907931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f8379-9c89-4bf8-9ce2-97afc11e1720 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:27.908427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f8379-9c89-4bf8-9ce2-97afc11e1720 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:27.908536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f8379-9c89-4bf8-9ce2-97afc11e1720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:27.927286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-207f8379-9c89-4bf8-9ce2-97afc11e1720 None None] MFA Rules not processed for user `f928852d90b94ac5979e4a7806896b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:27.928468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-207f8379-9c89-4bf8-9ce2-97afc11e1720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:27.935245 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1304/2591] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:56:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:28.247593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f338e-89b5-4558-9697-42d5fa410d2c service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:28.257269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f338e-89b5-4558-9697-42d5fa410d2c service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:28.260842 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed0d2a-1759-4bf9-b545-1d1af63048d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:28.261437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed0d2a-1759-4bf9-b545-1d1af63048d0 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:28.261743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed0d2a-1759-4bf9-b545-1d1af63048d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:28.262020 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed0d2a-1759-4bf9-b545-1d1af63048d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:28.263729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d9ed0d2a-1759-4bf9-b545-1d1af63048d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:28.273063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed0d2a-1759-4bf9-b545-1d1af63048d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:28.274508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed0d2a-1759-4bf9-b545-1d1af63048d0 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:28.282266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1288/2592] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:28.725418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f8379-9c89-4bf8-9ce2-97afc11e1720 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:28.726977 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-207f8379-9c89-4bf8-9ce2-97afc11e1720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:28.797886 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f8379-9c89-4bf8-9ce2-97afc11e1720 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:28.802913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c4bd5-11eb-48e8-9c39-fa6313b850a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:28.803523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4bd5-11eb-48e8-9c39-fa6313b850a8 service placement] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:28.803812 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4bd5-11eb-48e8-9c39-fa6313b850a8 service placement] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:28.804088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4bd5-11eb-48e8-9c39-fa6313b850a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:28.805860 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-007c4bd5-11eb-48e8-9c39-fa6313b850a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:28.818488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c4bd5-11eb-48e8-9c39-fa6313b850a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:28.820102 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c4bd5-11eb-48e8-9c39-fa6313b850a8 service placement] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:28.830813 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1305/2593] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:30.705296 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed0d2a-1759-4bf9-b545-1d1af63048d0 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:30.714557 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed0d2a-1759-4bf9-b545-1d1af63048d0 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:30.718282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3da3c-b142-4485-88c0-b80b175b10eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:30.718887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3da3c-b142-4485-88c0-b80b175b10eb service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:30.719741 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3da3c-b142-4485-88c0-b80b175b10eb service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:30.720016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3da3c-b142-4485-88c0-b80b175b10eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:30.721822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a6a3da3c-b142-4485-88c0-b80b175b10eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:30.732272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3da3c-b142-4485-88c0-b80b175b10eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:30.733738 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3da3c-b142-4485-88c0-b80b175b10eb service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:30.742329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1289/2594] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:31.398466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c4bd5-11eb-48e8-9c39-fa6313b850a8 service placement] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:31.402136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-007c4bd5-11eb-48e8-9c39-fa6313b850a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:31.418796 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c4bd5-11eb-48e8-9c39-fa6313b850a8 service placement] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:31.424889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3b532-f033-4aeb-888f-e083599f460b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:31.425495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3b532-f033-4aeb-888f-e083599f460b service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:31.425807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3b532-f033-4aeb-888f-e083599f460b service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:31.426085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3b532-f033-4aeb-888f-e083599f460b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:31.427805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-0ff3b532-f033-4aeb-888f-e083599f460b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:31.441020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3b532-f033-4aeb-888f-e083599f460b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:31.442656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3b532-f033-4aeb-888f-e083599f460b service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:31.453545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1306/2595] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:40.738559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=12 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:41.448807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=41 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:46.082724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3da3c-b142-4485-88c0-b80b175b10eb service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:46.096222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3da3c-b142-4485-88c0-b80b175b10eb service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:46.102593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc027af-eb9f-40ac-bb9e-2346c168a36e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:46.103195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc027af-eb9f-40ac-bb9e-2346c168a36e service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:46.103474 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc027af-eb9f-40ac-bb9e-2346c168a36e service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:46.103745 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc027af-eb9f-40ac-bb9e-2346c168a36e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:46.105499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7cc027af-eb9f-40ac-bb9e-2346c168a36e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:46.118447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc027af-eb9f-40ac-bb9e-2346c168a36e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:46.120071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc027af-eb9f-40ac-bb9e-2346c168a36e service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:46.131003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1290/2596] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:46.888934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3b532-f033-4aeb-888f-e083599f460b service nova] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:46.898696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3b532-f033-4aeb-888f-e083599f460b service nova] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:46.902284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82898cb6-0e1d-499e-93bf-31f7b6cab747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:46.902870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82898cb6-0e1d-499e-93bf-31f7b6cab747 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:46.903144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82898cb6-0e1d-499e-93bf-31f7b6cab747 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:46.903411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82898cb6-0e1d-499e-93bf-31f7b6cab747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:46.905136 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-82898cb6-0e1d-499e-93bf-31f7b6cab747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:46.967246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82898cb6-0e1d-499e-93bf-31f7b6cab747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:46.968665 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82898cb6-0e1d-499e-93bf-31f7b6cab747 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:46.978268 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1307/2597] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:49.074041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc027af-eb9f-40ac-bb9e-2346c168a36e service nova] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:49.086106 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc027af-eb9f-40ac-bb9e-2346c168a36e service nova] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:49.091152 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636ccb1-47d9-4914-b7f6-42ec50b34bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:49.092285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636ccb1-47d9-4914-b7f6-42ec50b34bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:49.093031 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636ccb1-47d9-4914-b7f6-42ec50b34bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:49.093788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636ccb1-47d9-4914-b7f6-42ec50b34bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:49.096881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1636ccb1-47d9-4914-b7f6-42ec50b34bf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:49.163433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636ccb1-47d9-4914-b7f6-42ec50b34bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:49.164872 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636ccb1-47d9-4914-b7f6-42ec50b34bf9 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:49.174956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1291/2598] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:50.243850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82898cb6-0e1d-499e-93bf-31f7b6cab747 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:50.245523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-82898cb6-0e1d-499e-93bf-31f7b6cab747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:50.262464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82898cb6-0e1d-499e-93bf-31f7b6cab747 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:50.267163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fe157-1cef-4bc5-ad35-3394154e9af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:50.267871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fe157-1cef-4bc5-ad35-3394154e9af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:50.268211 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fe157-1cef-4bc5-ad35-3394154e9af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:50.268575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fe157-1cef-4bc5-ad35-3394154e9af1 admin admin] PATH_INFO: `/v3/users/3c0b7528b0ff4b09923a0e951d052ed5` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:50.293268 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fe157-1cef-4bc5-ad35-3394154e9af1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c0b7528b0ff4b09923a0e951d052ed5)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:50.295703 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fe157-1cef-4bc5-ad35-3394154e9af1 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:50.348339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d06fe157-1cef-4bc5-ad35-3394154e9af1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c0b7528b0ff4b09923a0e951d052ed5'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:50.348925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d06fe157-1cef-4bc5-ad35-3394154e9af1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c0b7528b0ff4b09923a0e951d052ed5'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:50.356065 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d06fe157-1cef-4bc5-ad35-3394154e9af1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c0b7528b0ff4b09923a0e951d052ed5'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:50.356775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d06fe157-1cef-4bc5-ad35-3394154e9af1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c0b7528b0ff4b09923a0e951d052ed5'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:50.368663 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-d06fe157-1cef-4bc5-ad35-3394154e9af1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c0b7528b0ff4b09923a0e951d052ed5'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:50.383595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1308/2599] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:56:50 2022] DELETE /identity/v3/users/3c0b7528b0ff4b09923a0e951d052ed5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:56:50.568859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636ccb1-47d9-4914-b7f6-42ec50b34bf9 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:50.580860 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636ccb1-47d9-4914-b7f6-42ec50b34bf9 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:50.646023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:50.646685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:50.647033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:50.647392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2 admin admin] PATH_INFO: `/v3/projects/9bd782f79ed84b81a998ae437af2a6cb` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:50.660094 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd782f79ed84b81a998ae437af2a6cb)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:50.662364 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:50.714872 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd782f79ed84b81a998ae437af2a6cb', 'request_id': 'req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:50.721739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd782f79ed84b81a998ae437af2a6cb', 'request_id': 'req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:50.727592 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd782f79ed84b81a998ae437af2a6cb', 'request_id': 'req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:50.741245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2 admin admin] The token cache is being invalidate because project 9bd782f79ed84b81a998ae437af2a6cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:56:50.741567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:50.742236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8eebe9bf-0b72-46e9-a211-8b7123bb94c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:56:50.743729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1292/2600] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:56:50 2022] DELETE /identity/v3/projects/9bd782f79ed84b81a998ae437af2a6cb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:56:50.749076 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc5d63-0185-48c2-aee4-4865de549e07 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:50.749451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc5d63-0185-48c2-aee4-4865de549e07 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:50.749828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc5d63-0185-48c2-aee4-4865de549e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:50.772005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-48bc5d63-0185-48c2-aee4-4865de549e07 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:50.773202 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-48bc5d63-0185-48c2-aee4-4865de549e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:50.820798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:50.834561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1309/2601] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:50.842702 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323366-0631-48be-9755-52fc7e4e162f None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:50.842986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323366-0631-48be-9755-52fc7e4e162f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:50.843253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323366-0631-48be-9755-52fc7e4e162f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:50.850956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:50.864627 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-94323366-0631-48be-9755-52fc7e4e162f None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:50.865933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-94323366-0631-48be-9755-52fc7e4e162f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:50.873981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1293/2602] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:50.878788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc5d63-0185-48c2-aee4-4865de549e07 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:50.888547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc5d63-0185-48c2-aee4-4865de549e07 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:50.893250 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102158ec-f7b4-4084-8e0f-4cfedb1d3557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:50.894543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102158ec-f7b4-4084-8e0f-4cfedb1d3557 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:50.894843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102158ec-f7b4-4084-8e0f-4cfedb1d3557 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:50.895115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102158ec-f7b4-4084-8e0f-4cfedb1d3557 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:50.895765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102158ec-f7b4-4084-8e0f-4cfedb1d3557 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:56:50.896119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102158ec-f7b4-4084-8e0f-4cfedb1d3557 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:50.897784 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102158ec-f7b4-4084-8e0f-4cfedb1d3557 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:50.912794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1310/2603] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:56:50.917524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94323366-0631-48be-9755-52fc7e4e162f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:50.927398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94323366-0631-48be-9755-52fc7e4e162f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:50.932607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfab9ae-c50b-4f50-8241-688f18e06a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:50.933204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfab9ae-c50b-4f50-8241-688f18e06a25 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:50.933486 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfab9ae-c50b-4f50-8241-688f18e06a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:50.933793 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfab9ae-c50b-4f50-8241-688f18e06a25 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:50.934419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfab9ae-c50b-4f50-8241-688f18e06a25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:50.941671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfab9ae-c50b-4f50-8241-688f18e06a25 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:50.984643 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1294/2604] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:56:50 2022] POST /identity/v3/projects => generated 365 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:56:50.985852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102158ec-f7b4-4084-8e0f-4cfedb1d3557 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:50.996439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102158ec-f7b4-4084-8e0f-4cfedb1d3557 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:51.053630 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d17ec2-5ee5-4e4a-a835-24517e265ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:51.054532 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d17ec2-5ee5-4e4a-a835-24517e265ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:51.054815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d17ec2-5ee5-4e4a-a835-24517e265ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:51.055092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d17ec2-5ee5-4e4a-a835-24517e265ac8 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:51.055726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d17ec2-5ee5-4e4a-a835-24517e265ac8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:51.057657 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d17ec2-5ee5-4e4a-a835-24517e265ac8 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:51.083209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1311/2605] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:56:50 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:56:51.087480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfab9ae-c50b-4f50-8241-688f18e06a25 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:51.099026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfab9ae-c50b-4f50-8241-688f18e06a25 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:51.104590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1716aff-7c2b-4606-875a-d9aebd561b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:51.105378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1716aff-7c2b-4606-875a-d9aebd561b42 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:51.105838 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1716aff-7c2b-4606-875a-d9aebd561b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:51.106795 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1716aff-7c2b-4606-875a-d9aebd561b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:51.107796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1716aff-7c2b-4606-875a-d9aebd561b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:51.109563 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1716aff-7c2b-4606-875a-d9aebd561b42 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:51.123112 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1295/2606] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:56:51.127382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d17ec2-5ee5-4e4a-a835-24517e265ac8 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:51.137650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d17ec2-5ee5-4e4a-a835-24517e265ac8 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:51.142253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e0209-7480-4a6d-a388-3103eb029ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:51.142880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e0209-7480-4a6d-a388-3103eb029ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:51.143157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e0209-7480-4a6d-a388-3103eb029ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:51.143430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e0209-7480-4a6d-a388-3103eb029ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:51.144100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e0209-7480-4a6d-a388-3103eb029ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:51.145702 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e0209-7480-4a6d-a388-3103eb029ea7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:51.157706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1312/2607] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:56:51.162398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1716aff-7c2b-4606-875a-d9aebd561b42 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:51.177285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1716aff-7c2b-4606-875a-d9aebd561b42 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:51.181961 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455aed4-5815-4941-8687-430ee41d364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:51.182609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455aed4-5815-4941-8687-430ee41d364a admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:51.182887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455aed4-5815-4941-8687-430ee41d364a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:51.183160 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455aed4-5815-4941-8687-430ee41d364a admin admin] PATH_INFO: `/v3/projects/df573c7a7f7a410ab53bea4dbfcc6eb4/users/cfc4338079af4386bec3cd755dbb87f3/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:51.198726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455aed4-5815-4941-8687-430ee41d364a admin admin] RBAC: Authorizing `identity:create_grant(project_id=df573c7a7f7a410ab53bea4dbfcc6eb4, user_id=cfc4338079af4386bec3cd755dbb87f3, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:51.201660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455aed4-5815-4941-8687-430ee41d364a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:51.215251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1296/2608] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:56:51 2022] PUT /identity/v3/projects/df573c7a7f7a410ab53bea4dbfcc6eb4/users/cfc4338079af4386bec3cd755dbb87f3/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:56:51.227412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404339c3-4236-45c8-a8a1-52f1d2233def None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:51.227842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404339c3-4236-45c8-a8a1-52f1d2233def None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:51.228219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404339c3-4236-45c8-a8a1-52f1d2233def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:51.264481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-404339c3-4236-45c8-a8a1-52f1d2233def None None] MFA Rules not processed for user `cfc4338079af4386bec3cd755dbb87f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:56:51.265920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-404339c3-4236-45c8-a8a1-52f1d2233def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:51.340163 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1313/2609] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:56:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:56:51.351534 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455aed4-5815-4941-8687-430ee41d364a admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:51.353608 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7455aed4-5815-4941-8687-430ee41d364a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:51.415573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455aed4-5815-4941-8687-430ee41d364a admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:51.419270 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60a431-e18d-4c7e-9463-f4c8ca8325e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:51.419890 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60a431-e18d-4c7e-9463-f4c8ca8325e0 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:51.420171 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60a431-e18d-4c7e-9463-f4c8ca8325e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:51.420442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60a431-e18d-4c7e-9463-f4c8ca8325e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:51.429021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60a431-e18d-4c7e-9463-f4c8ca8325e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:51.430491 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60a431-e18d-4c7e-9463-f4c8ca8325e0 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:51.439901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1297/2610] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:55.232041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404339c3-4236-45c8-a8a1-52f1d2233def None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:55.233524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-404339c3-4236-45c8-a8a1-52f1d2233def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:55.290470 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404339c3-4236-45c8-a8a1-52f1d2233def None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:55.294037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741a838-e9bd-4740-959c-bd7d2390a951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:55.294645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741a838-e9bd-4740-959c-bd7d2390a951 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:55.294923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741a838-e9bd-4740-959c-bd7d2390a951 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:55.295193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741a838-e9bd-4740-959c-bd7d2390a951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:55.296947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4741a838-e9bd-4740-959c-bd7d2390a951 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:55.302363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:56:55.356828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741a838-e9bd-4740-959c-bd7d2390a951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:55.358286 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741a838-e9bd-4740-959c-bd7d2390a951 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:55.367508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1314/2611] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:56:56.283493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60a431-e18d-4c7e-9463-f4c8ca8325e0 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:56:56.285459 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-9a60a431-e18d-4c7e-9463-f4c8ca8325e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:56.388163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60a431-e18d-4c7e-9463-f4c8ca8325e0 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:56:56.393783 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05968715-097a-4052-afbf-9ddb164d3cd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:56:56.394936 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05968715-097a-4052-afbf-9ddb164d3cd6 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:56:56.395221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05968715-097a-4052-afbf-9ddb164d3cd6 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:56:56.395493 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05968715-097a-4052-afbf-9ddb164d3cd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:56:56.397266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-05968715-097a-4052-afbf-9ddb164d3cd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:56:56.412089 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05968715-097a-4052-afbf-9ddb164d3cd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:56:56.414350 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05968715-097a-4052-afbf-9ddb164d3cd6 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:56:56.435019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1298/2612] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:56:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:00.035179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741a838-e9bd-4740-959c-bd7d2390a951 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:00.119519 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741a838-e9bd-4740-959c-bd7d2390a951 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:00.124712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:00.125352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:00.125626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:00.125943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] PATH_INFO: `/v3/users/cfc4338079af4386bec3cd755dbb87f3` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:00.127192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc4338079af4386bec3cd755dbb87f3)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:00.129216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:00.178805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc4338079af4386bec3cd755dbb87f3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:00.179467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc4338079af4386bec3cd755dbb87f3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:00.187131 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc4338079af4386bec3cd755dbb87f3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:00.187828 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc4338079af4386bec3cd755dbb87f3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:00.202002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc4338079af4386bec3cd755dbb87f3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:00.216283 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1315/2613] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:00 2022] DELETE /identity/v3/users/cfc4338079af4386bec3cd755dbb87f3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:00.220876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4307ae2-a4f4-4c49-bfd6-705b04931981 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:00.221378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4307ae2-a4f4-4c49-bfd6-705b04931981 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:00.221521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4307ae2-a4f4-4c49-bfd6-705b04931981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:00.241216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d4307ae2-a4f4-4c49-bfd6-705b04931981 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:00.242448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d4307ae2-a4f4-4c49-bfd6-705b04931981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:00.255040 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:00.303920 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1299/2614] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:00.313542 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:00.314967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:00.343610 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:00.375396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82092b3f-7209-4e55-8225-9242c469fd50 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:00.379942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5596c1f-ccbb-4bd6-9ae7-607ad32f15de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:00.380640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5596c1f-ccbb-4bd6-9ae7-607ad32f15de service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:00.380999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5596c1f-ccbb-4bd6-9ae7-607ad32f15de service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:00.381320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5596c1f-ccbb-4bd6-9ae7-607ad32f15de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:00.391247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5596c1f-ccbb-4bd6-9ae7-607ad32f15de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:00.392686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5596c1f-ccbb-4bd6-9ae7-607ad32f15de service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:00.403076 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1316/2615] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:00.675835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4307ae2-a4f4-4c49-bfd6-705b04931981 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:00.677905 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d4307ae2-a4f4-4c49-bfd6-705b04931981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:00.702100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4307ae2-a4f4-4c49-bfd6-705b04931981 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:00.708903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:00.709548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:00.709867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:00.710142 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d admin admin] PATH_INFO: `/v3/projects/df573c7a7f7a410ab53bea4dbfcc6eb4` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:00.711281 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d admin admin] RBAC: Authorizing `identity:delete_project(project_id=df573c7a7f7a410ab53bea4dbfcc6eb4)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:00.713198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:00.760251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df573c7a7f7a410ab53bea4dbfcc6eb4', 'request_id': 'req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:00.766354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df573c7a7f7a410ab53bea4dbfcc6eb4', 'request_id': 'req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:00.772251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df573c7a7f7a410ab53bea4dbfcc6eb4', 'request_id': 'req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:00.783883 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d admin admin] The token cache is being invalidate because project df573c7a7f7a410ab53bea4dbfcc6eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:00.784177 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:00.784828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-34e52b9c-2e45-4aa3-a6e9-9e02b365b10d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:00.786913 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1300/2616] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:00 2022] DELETE /identity/v3/projects/df573c7a7f7a410ab53bea4dbfcc6eb4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:00.791949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27f59c-5023-41a5-ab8b-1cd007850a12 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:00.792237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27f59c-5023-41a5-ab8b-1cd007850a12 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:00.792514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27f59c-5023-41a5-ab8b-1cd007850a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:00.811508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-bd27f59c-5023-41a5-ab8b-1cd007850a12 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:00.812711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-bd27f59c-5023-41a5-ab8b-1cd007850a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:00.870775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1317/2617] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:00.878623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45f8d3-2f19-466d-bcb8-9a0efc53d349 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:00.878912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45f8d3-2f19-466d-bcb8-9a0efc53d349 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:00.879191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45f8d3-2f19-466d-bcb8-9a0efc53d349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:00.899169 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-2d45f8d3-2f19-466d-bcb8-9a0efc53d349 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:00.900370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2d45f8d3-2f19-466d-bcb8-9a0efc53d349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:00.908259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1301/2618] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:00.912969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27f59c-5023-41a5-ab8b-1cd007850a12 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:00.922475 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27f59c-5023-41a5-ab8b-1cd007850a12 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:00.926968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74703230-1a82-4784-afdb-8cc0234482d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:00.927558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74703230-1a82-4784-afdb-8cc0234482d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:00.927832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74703230-1a82-4784-afdb-8cc0234482d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:00.928101 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74703230-1a82-4784-afdb-8cc0234482d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:00.928733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74703230-1a82-4784-afdb-8cc0234482d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:57:00.929032 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74703230-1a82-4784-afdb-8cc0234482d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:00.930661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74703230-1a82-4784-afdb-8cc0234482d4 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:00.941967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1318/2619] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:57:00.946198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45f8d3-2f19-466d-bcb8-9a0efc53d349 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:00.955305 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45f8d3-2f19-466d-bcb8-9a0efc53d349 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:00.960763 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c846c53-461d-4c20-85bc-ecf02b4f9d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:00.961357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c846c53-461d-4c20-85bc-ecf02b4f9d70 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:00.961637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c846c53-461d-4c20-85bc-ecf02b4f9d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:00.961942 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c846c53-461d-4c20-85bc-ecf02b4f9d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:00.962580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c846c53-461d-4c20-85bc-ecf02b4f9d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:00.964467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c846c53-461d-4c20-85bc-ecf02b4f9d70 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:00.991076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1302/2620] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:00 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:00.995275 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74703230-1a82-4784-afdb-8cc0234482d4 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:01.005141 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74703230-1a82-4784-afdb-8cc0234482d4 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:01.058722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096eb01c-0d30-4e31-a31c-af04ca6563bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:01.059372 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096eb01c-0d30-4e31-a31c-af04ca6563bd admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.059584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096eb01c-0d30-4e31-a31c-af04ca6563bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.059851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096eb01c-0d30-4e31-a31c-af04ca6563bd admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.060478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096eb01c-0d30-4e31-a31c-af04ca6563bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:01.062458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096eb01c-0d30-4e31-a31c-af04ca6563bd admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:01.084415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1319/2621] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:00 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:01.088424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c846c53-461d-4c20-85bc-ecf02b4f9d70 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:01.098612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c846c53-461d-4c20-85bc-ecf02b4f9d70 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:01.103247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31fe03-7a43-4f33-a1fd-18ee67fe72d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:01.103840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31fe03-7a43-4f33-a1fd-18ee67fe72d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.104119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31fe03-7a43-4f33-a1fd-18ee67fe72d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.104390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31fe03-7a43-4f33-a1fd-18ee67fe72d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.105026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31fe03-7a43-4f33-a1fd-18ee67fe72d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:01.106870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31fe03-7a43-4f33-a1fd-18ee67fe72d4 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:01.119946 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1303/2622] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:01.121167 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096eb01c-0d30-4e31-a31c-af04ca6563bd admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:01.131779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096eb01c-0d30-4e31-a31c-af04ca6563bd admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:01.136392 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90208de3-efec-43e0-92b4-37c770f6595f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:01.136995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90208de3-efec-43e0-92b4-37c770f6595f admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.137257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90208de3-efec-43e0-92b4-37c770f6595f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.137529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90208de3-efec-43e0-92b4-37c770f6595f admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.140426 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90208de3-efec-43e0-92b4-37c770f6595f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:01.140426 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90208de3-efec-43e0-92b4-37c770f6595f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:01.157275 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1320/2623] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:01.161728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31fe03-7a43-4f33-a1fd-18ee67fe72d4 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:01.171737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31fe03-7a43-4f33-a1fd-18ee67fe72d4 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:01.176432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70df5c-e2dd-4831-9999-3ee7867923ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:01.177179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70df5c-e2dd-4831-9999-3ee7867923ca admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.177532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70df5c-e2dd-4831-9999-3ee7867923ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.177912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70df5c-e2dd-4831-9999-3ee7867923ca admin admin] PATH_INFO: `/v3/projects/d6e8bd05f2fd43fd95538ca398196fba/users/ceffaee376114ada86eb758ba134885f/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.197857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70df5c-e2dd-4831-9999-3ee7867923ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e8bd05f2fd43fd95538ca398196fba, user_id=ceffaee376114ada86eb758ba134885f, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:01.200640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70df5c-e2dd-4831-9999-3ee7867923ca admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:01.215225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1304/2624] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:01 2022] PUT /identity/v3/projects/d6e8bd05f2fd43fd95538ca398196fba/users/ceffaee376114ada86eb758ba134885f/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:01.225853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e55a86-08da-4dce-a6fc-9201ae65837d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.226182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e55a86-08da-4dce-a6fc-9201ae65837d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.226464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e55a86-08da-4dce-a6fc-9201ae65837d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.256411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-67e55a86-08da-4dce-a6fc-9201ae65837d None None] MFA Rules not processed for user `ceffaee376114ada86eb758ba134885f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:01.257632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-67e55a86-08da-4dce-a6fc-9201ae65837d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:01.279914 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:01.337898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1321/2625] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:01.339237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70df5c-e2dd-4831-9999-3ee7867923ca admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:01.380257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e55a86-08da-4dce-a6fc-9201ae65837d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:01.381830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-67e55a86-08da-4dce-a6fc-9201ae65837d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:01.389590 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:01.412567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70df5c-e2dd-4831-9999-3ee7867923ca admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:01.417268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1e7f7-c67c-4255-9db3-146da717b4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:01.417898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1e7f7-c67c-4255-9db3-146da717b4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.418176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1e7f7-c67c-4255-9db3-146da717b4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.418448 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1e7f7-c67c-4255-9db3-146da717b4d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.419081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1e7f7-c67c-4255-9db3-146da717b4d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:01.420976 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1e7f7-c67c-4255-9db3-146da717b4d0 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:01.440004 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e55a86-08da-4dce-a6fc-9201ae65837d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:01.444528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca84eda-bf3c-4593-800d-a15aff8c8fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:01.445140 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca84eda-bf3c-4593-800d-a15aff8c8fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.445427 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca84eda-bf3c-4593-800d-a15aff8c8fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.445713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca84eda-bf3c-4593-800d-a15aff8c8fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.447520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2ca84eda-bf3c-4593-800d-a15aff8c8fe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:01.452511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1305/2626] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:01 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:01.453606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1e7f7-c67c-4255-9db3-146da717b4d0 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:01.460141 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca84eda-bf3c-4593-800d-a15aff8c8fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:01.461578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca84eda-bf3c-4593-800d-a15aff8c8fe3 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:01.471269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1322/2627] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:57:01.515357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1e7f7-c67c-4255-9db3-146da717b4d0 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:01.520404 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe368c-6a78-46fb-bfd4-558682e14457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:01.521015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe368c-6a78-46fb-bfd4-558682e14457 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.521292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe368c-6a78-46fb-bfd4-558682e14457 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.521573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe368c-6a78-46fb-bfd4-558682e14457 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.522224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe368c-6a78-46fb-bfd4-558682e14457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:01.524133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe368c-6a78-46fb-bfd4-558682e14457 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:01.544896 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1306/2628] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:01 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:01.548913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca84eda-bf3c-4593-800d-a15aff8c8fe3 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:01.559197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca84eda-bf3c-4593-800d-a15aff8c8fe3 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:01.563890 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3d8d5-7c71-4504-8b81-985aa699fb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:01.564489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3d8d5-7c71-4504-8b81-985aa699fb00 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.564769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3d8d5-7c71-4504-8b81-985aa699fb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.565042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3d8d5-7c71-4504-8b81-985aa699fb00 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.565680 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3d8d5-7c71-4504-8b81-985aa699fb00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:01.567879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3d8d5-7c71-4504-8b81-985aa699fb00 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:01.577951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1323/2629] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:01.582311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe368c-6a78-46fb-bfd4-558682e14457 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:01.592242 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe368c-6a78-46fb-bfd4-558682e14457 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:01.596786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c16b4a-f8d7-4628-9b03-9281e4768a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:01.597433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c16b4a-f8d7-4628-9b03-9281e4768a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.597732 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c16b4a-f8d7-4628-9b03-9281e4768a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.598008 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c16b4a-f8d7-4628-9b03-9281e4768a53 admin admin] PATH_INFO: `/v3/projects/ec09662a92a1451580e504c920a4a12c/users/e2d3d02d7b21477ca0abb173c78f81bd/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.612649 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c16b4a-f8d7-4628-9b03-9281e4768a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec09662a92a1451580e504c920a4a12c, user_id=e2d3d02d7b21477ca0abb173c78f81bd, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:01.615959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c16b4a-f8d7-4628-9b03-9281e4768a53 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:01.628989 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1307/2630] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:01 2022] PUT /identity/v3/projects/ec09662a92a1451580e504c920a4a12c/users/e2d3d02d7b21477ca0abb173c78f81bd/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:01.633515 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3d8d5-7c71-4504-8b81-985aa699fb00 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:01.643489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3d8d5-7c71-4504-8b81-985aa699fb00 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:01.708961 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feef4ab-5d7f-476a-9cfe-7b81b9b51d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:01.709512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feef4ab-5d7f-476a-9cfe-7b81b9b51d61 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.710448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feef4ab-5d7f-476a-9cfe-7b81b9b51d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.710832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feef4ab-5d7f-476a-9cfe-7b81b9b51d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.711377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feef4ab-5d7f-476a-9cfe-7b81b9b51d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:01.713012 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feef4ab-5d7f-476a-9cfe-7b81b9b51d61 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:01.723857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1324/2631] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:01.727863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c16b4a-f8d7-4628-9b03-9281e4768a53 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:01.739922 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c16b4a-f8d7-4628-9b03-9281e4768a53 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:01.745984 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967161eb-6ff6-417e-874a-cca44e3dea9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:01.746654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967161eb-6ff6-417e-874a-cca44e3dea9c admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.746941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967161eb-6ff6-417e-874a-cca44e3dea9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.747221 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967161eb-6ff6-417e-874a-cca44e3dea9c admin admin] PATH_INFO: `/v3/projects/ec09662a92a1451580e504c920a4a12c/users/e2d3d02d7b21477ca0abb173c78f81bd/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.749299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967161eb-6ff6-417e-874a-cca44e3dea9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec09662a92a1451580e504c920a4a12c, user_id=e2d3d02d7b21477ca0abb173c78f81bd, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:01.752042 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967161eb-6ff6-417e-874a-cca44e3dea9c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:01.763615 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1308/2632] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:01 2022] PUT /identity/v3/projects/ec09662a92a1451580e504c920a4a12c/users/e2d3d02d7b21477ca0abb173c78f81bd/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:57:01.775400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f98d9-b93a-4655-be86-8b1b2b982c29 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.775702 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f98d9-b93a-4655-be86-8b1b2b982c29 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.776690 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f98d9-b93a-4655-be86-8b1b2b982c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.806052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-f18f98d9-b93a-4655-be86-8b1b2b982c29 None None] MFA Rules not processed for user `e2d3d02d7b21477ca0abb173c78f81bd`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:01.807343 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f18f98d9-b93a-4655-be86-8b1b2b982c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:01.849046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:01.878315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1325/2633] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:01.889351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967161eb-6ff6-417e-874a-cca44e3dea9c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:01.891254 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-967161eb-6ff6-417e-874a-cca44e3dea9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:01.949304 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967161eb-6ff6-417e-874a-cca44e3dea9c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:01.953019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e36f86-a199-4cfa-b7fc-3db4bf9eb226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:01.953625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e36f86-a199-4cfa-b7fc-3db4bf9eb226 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:01.953929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e36f86-a199-4cfa-b7fc-3db4bf9eb226 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:01.954205 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e36f86-a199-4cfa-b7fc-3db4bf9eb226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:01.963584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e36f86-a199-4cfa-b7fc-3db4bf9eb226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:01.964949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e36f86-a199-4cfa-b7fc-3db4bf9eb226 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:01.974939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1309/2634] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:02.052322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f98d9-b93a-4655-be86-8b1b2b982c29 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:02.062195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f98d9-b93a-4655-be86-8b1b2b982c29 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:02.071641 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f0f71-5a2e-40b6-8ddb-4cede92a3845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:02.082442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f0f71-5a2e-40b6-8ddb-4cede92a3845 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:02.082736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f0f71-5a2e-40b6-8ddb-4cede92a3845 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:02.083019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f0f71-5a2e-40b6-8ddb-4cede92a3845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:02.147059 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f0f71-5a2e-40b6-8ddb-4cede92a3845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:02.148490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f0f71-5a2e-40b6-8ddb-4cede92a3845 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:02.157547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1326/2635] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:07.817672 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e36f86-a199-4cfa-b7fc-3db4bf9eb226 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:07.823025 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-81e36f86-a199-4cfa-b7fc-3db4bf9eb226 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:07.845208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:07.891301 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e36f86-a199-4cfa-b7fc-3db4bf9eb226 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:07.895860 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:07.896486 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:07.896770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:07.897048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] PATH_INFO: `/v3/users/dfe240ebc7c643358db4ae3a9ef132ea` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:07.898213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfe240ebc7c643358db4ae3a9ef132ea)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:07.900170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:07.946095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfe240ebc7c643358db4ae3a9ef132ea'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:07.946746 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfe240ebc7c643358db4ae3a9ef132ea'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:07.958843 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfe240ebc7c643358db4ae3a9ef132ea'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:07.965716 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfe240ebc7c643358db4ae3a9ef132ea'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:07.966367 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfe240ebc7c643358db4ae3a9ef132ea'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:07.979500 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1310/2636] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:07 2022] DELETE /identity/v3/users/dfe240ebc7c643358db4ae3a9ef132ea => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:08.156342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f0f71-5a2e-40b6-8ddb-4cede92a3845 service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:08.167866 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f0f71-5a2e-40b6-8ddb-4cede92a3845 service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:08.224619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:08.225241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:08.225522 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:08.225826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] PATH_INFO: `/v3/projects/4a75a59f42a74dee854ddc7b5f109143` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:08.226990 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a75a59f42a74dee854ddc7b5f109143)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:08.228904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:08.273381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a75a59f42a74dee854ddc7b5f109143', 'request_id': 'req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.279243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a75a59f42a74dee854ddc7b5f109143', 'request_id': 'req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.284723 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a75a59f42a74dee854ddc7b5f109143', 'request_id': 'req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.297047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] The token cache is being invalidate because project 4a75a59f42a74dee854ddc7b5f109143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:08.297337 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.297981 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.299429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1327/2637] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:08 2022] DELETE /identity/v3/projects/4a75a59f42a74dee854ddc7b5f109143 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:57:08.303526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:08.304964 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:08.317940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63aefe-611d-479c-bfa8-82e48c9942cc admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:08.373901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:08.374533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:08.374816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:08.375088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] PATH_INFO: `/v3/users/06f3c8346a7b48a0a0b9631a9794f85d` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:08.376245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=06f3c8346a7b48a0a0b9631a9794f85d)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:08.378189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:08.421081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:08.428499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06f3c8346a7b48a0a0b9631a9794f85d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.429435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06f3c8346a7b48a0a0b9631a9794f85d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.444232 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06f3c8346a7b48a0a0b9631a9794f85d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.451280 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06f3c8346a7b48a0a0b9631a9794f85d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.451951 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06f3c8346a7b48a0a0b9631a9794f85d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.468004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1311/2638] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:08 2022] DELETE /identity/v3/users/06f3c8346a7b48a0a0b9631a9794f85d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:08.672412 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:08.675033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:08.684638 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:08.688707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7783bfd-4a75-4d15-b34a-c77c43f13bf9 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:08.752336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:08.752989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:08.753269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:08.753546 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] PATH_INFO: `/v3/projects/91af61bce2c14978bec974811300f333` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:08.755326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] RBAC: Authorizing `identity:delete_project(project_id=91af61bce2c14978bec974811300f333)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:08.757277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:08.808931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91af61bce2c14978bec974811300f333', 'request_id': 'req-21f92ba7-a943-4045-90ef-8a56b6efa8db'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.820249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91af61bce2c14978bec974811300f333', 'request_id': 'req-21f92ba7-a943-4045-90ef-8a56b6efa8db'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.826632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91af61bce2c14978bec974811300f333', 'request_id': 'req-21f92ba7-a943-4045-90ef-8a56b6efa8db'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.841165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] The token cache is being invalidate because project 91af61bce2c14978bec974811300f333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:08.841463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.842172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:08.843911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1328/2639] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:08 2022] DELETE /identity/v3/projects/91af61bce2c14978bec974811300f333 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:10.175956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:10.177346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:10.233916 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9168a-424c-4996-95ee-17b2951093ad admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:10.281379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383d398-4f99-498b-bc06-45a4142560b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:10.282026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d398-4f99-498b-bc06-45a4142560b0 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:10.282311 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d398-4f99-498b-bc06-45a4142560b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:10.282585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d398-4f99-498b-bc06-45a4142560b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:10.284323 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1383d398-4f99-498b-bc06-45a4142560b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:10.364710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383d398-4f99-498b-bc06-45a4142560b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:10.366187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383d398-4f99-498b-bc06-45a4142560b0 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:10.380034 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:10.387566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1312/2640] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:13.577661 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:13.579168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:13.647186 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f92ba7-a943-4045-90ef-8a56b6efa8db admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:13.651909 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:13.652627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:13.652973 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:13.653306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] PATH_INFO: `/v3/users/ceffaee376114ada86eb758ba134885f` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:13.654559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceffaee376114ada86eb758ba134885f)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:13.656572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:13.709307 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:13.719237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceffaee376114ada86eb758ba134885f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:13.721409 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceffaee376114ada86eb758ba134885f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:13.731257 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceffaee376114ada86eb758ba134885f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:13.731984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceffaee376114ada86eb758ba134885f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:13.746237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceffaee376114ada86eb758ba134885f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:13.759561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1329/2641] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:13 2022] DELETE /identity/v3/users/ceffaee376114ada86eb758ba134885f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:13.764085 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95198aed-9f05-4763-b672-5536bb3064a3 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:13.764583 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95198aed-9f05-4763-b672-5536bb3064a3 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:13.764699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95198aed-9f05-4763-b672-5536bb3064a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:13.784459 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-95198aed-9f05-4763-b672-5536bb3064a3 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:13.785846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-95198aed-9f05-4763-b672-5536bb3064a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:13.847696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1313/2642] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:13.857533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:13.859200 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:13.919107 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff53b37-fbd5-4bbf-8e9d-1e6497da8e4b admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:13.924089 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657b9e0-c63b-47da-b8a0-d84704043882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:13.924581 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657b9e0-c63b-47da-b8a0-d84704043882 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:13.924937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657b9e0-c63b-47da-b8a0-d84704043882 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:13.925277 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657b9e0-c63b-47da-b8a0-d84704043882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:13.936139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657b9e0-c63b-47da-b8a0-d84704043882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:13.937504 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657b9e0-c63b-47da-b8a0-d84704043882 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:13.948441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1330/2643] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:14.292748 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95198aed-9f05-4763-b672-5536bb3064a3 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:14.294811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-95198aed-9f05-4763-b672-5536bb3064a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:14.310231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95198aed-9f05-4763-b672-5536bb3064a3 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:14.315679 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:14.317203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:14.317492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:14.317919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] PATH_INFO: `/v3/projects/d6e8bd05f2fd43fd95538ca398196fba` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:14.319737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e8bd05f2fd43fd95538ca398196fba)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:14.321681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:14.375088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e8bd05f2fd43fd95538ca398196fba', 'request_id': 'req-263353be-281b-413e-b64a-2677077cde81'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:14.382905 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e8bd05f2fd43fd95538ca398196fba', 'request_id': 'req-263353be-281b-413e-b64a-2677077cde81'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:14.390318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e8bd05f2fd43fd95538ca398196fba', 'request_id': 'req-263353be-281b-413e-b64a-2677077cde81'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:14.406148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] The token cache is being invalidate because project d6e8bd05f2fd43fd95538ca398196fba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:14.407299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:14.408559 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:14.413528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1314/2644] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:14 2022] DELETE /identity/v3/projects/d6e8bd05f2fd43fd95538ca398196fba => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:14.414308 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657b9e0-c63b-47da-b8a0-d84704043882 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:14.424732 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657b9e0-c63b-47da-b8a0-d84704043882 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:14.491300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf378b9-242d-4b3c-889a-f613f6a0ba16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:14.491931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf378b9-242d-4b3c-889a-f613f6a0ba16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:14.492214 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf378b9-242d-4b3c-889a-f613f6a0ba16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:14.492491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf378b9-242d-4b3c-889a-f613f6a0ba16 admin admin] PATH_INFO: `/v3/users/e2d3d02d7b21477ca0abb173c78f81bd` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:14.493623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf378b9-242d-4b3c-889a-f613f6a0ba16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d3d02d7b21477ca0abb173c78f81bd)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:14.495583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf378b9-242d-4b3c-889a-f613f6a0ba16 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:14.543411 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6cf378b9-242d-4b3c-889a-f613f6a0ba16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d3d02d7b21477ca0abb173c78f81bd'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:14.544083 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6cf378b9-242d-4b3c-889a-f613f6a0ba16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d3d02d7b21477ca0abb173c78f81bd'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:14.551390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6cf378b9-242d-4b3c-889a-f613f6a0ba16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d3d02d7b21477ca0abb173c78f81bd'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:14.552131 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6cf378b9-242d-4b3c-889a-f613f6a0ba16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d3d02d7b21477ca0abb173c78f81bd'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:14.554157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:14.568002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6cf378b9-242d-4b3c-889a-f613f6a0ba16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d3d02d7b21477ca0abb173c78f81bd'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:14.582298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1331/2645] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:14 2022] DELETE /identity/v3/users/e2d3d02d7b21477ca0abb173c78f81bd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:57:14.889731 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:14.891326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:14.906492 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263353be-281b-413e-b64a-2677077cde81 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:14.966290 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56921f36-727b-4beb-9709-0451255678c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:14.966928 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56921f36-727b-4beb-9709-0451255678c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:14.967206 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56921f36-727b-4beb-9709-0451255678c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:14.967478 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56921f36-727b-4beb-9709-0451255678c9 admin admin] PATH_INFO: `/v3/projects/ec09662a92a1451580e504c920a4a12c` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:14.968641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56921f36-727b-4beb-9709-0451255678c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec09662a92a1451580e504c920a4a12c)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:14.970644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56921f36-727b-4beb-9709-0451255678c9 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:14.997317 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:15.021056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-56921f36-727b-4beb-9709-0451255678c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec09662a92a1451580e504c920a4a12c', 'request_id': 'req-56921f36-727b-4beb-9709-0451255678c9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:15.027314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-56921f36-727b-4beb-9709-0451255678c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec09662a92a1451580e504c920a4a12c', 'request_id': 'req-56921f36-727b-4beb-9709-0451255678c9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:15.033888 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-56921f36-727b-4beb-9709-0451255678c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec09662a92a1451580e504c920a4a12c', 'request_id': 'req-56921f36-727b-4beb-9709-0451255678c9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:15.047530 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-56921f36-727b-4beb-9709-0451255678c9 admin admin] The token cache is being invalidate because project ec09662a92a1451580e504c920a4a12c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:15.047912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-56921f36-727b-4beb-9709-0451255678c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:15.048551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-56921f36-727b-4beb-9709-0451255678c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:15.050126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1315/2646] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:14 2022] DELETE /identity/v3/projects/ec09662a92a1451580e504c920a4a12c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:57:15.058925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404c4f4-0033-4d4a-9769-416626e0f1d9 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.059219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404c4f4-0033-4d4a-9769-416626e0f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.059498 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d404c4f4-0033-4d4a-9769-416626e0f1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.080995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-d404c4f4-0033-4d4a-9769-416626e0f1d9 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:15.088196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d404c4f4-0033-4d4a-9769-416626e0f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:15.151449 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1332/2647] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:15.161471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6b636-5d1b-455f-b155-31f7179cc324 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.161471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6b636-5d1b-455f-b155-31f7179cc324 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.161471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6b636-5d1b-455f-b155-31f7179cc324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.181844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f8b6b636-5d1b-455f-b155-31f7179cc324 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:15.183050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f8b6b636-5d1b-455f-b155-31f7179cc324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:15.191089 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1316/2648] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:15.195382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404c4f4-0033-4d4a-9769-416626e0f1d9 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:15.205349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d404c4f4-0033-4d4a-9769-416626e0f1d9 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:15.210186 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01217c9-f5c0-4a4b-96cd-975d67b7d944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:15.210776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01217c9-f5c0-4a4b-96cd-975d67b7d944 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.211053 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01217c9-f5c0-4a4b-96cd-975d67b7d944 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.211320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01217c9-f5c0-4a4b-96cd-975d67b7d944 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.212006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01217c9-f5c0-4a4b-96cd-975d67b7d944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:57:15.212311 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01217c9-f5c0-4a4b-96cd-975d67b7d944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:15.213960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01217c9-f5c0-4a4b-96cd-975d67b7d944 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:15.225756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1333/2649] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:57:15.230004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6b636-5d1b-455f-b155-31f7179cc324 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:15.240024 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6b636-5d1b-455f-b155-31f7179cc324 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:15.244771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fde53d-546c-457e-811c-3b0c251f6a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:15.245364 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fde53d-546c-457e-811c-3b0c251f6a78 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.245649 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fde53d-546c-457e-811c-3b0c251f6a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.245956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fde53d-546c-457e-811c-3b0c251f6a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.246596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fde53d-546c-457e-811c-3b0c251f6a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:15.248501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fde53d-546c-457e-811c-3b0c251f6a78 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:15.274978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1317/2650] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:15 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:57:15.279130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01217c9-f5c0-4a4b-96cd-975d67b7d944 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:15.289394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01217c9-f5c0-4a4b-96cd-975d67b7d944 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:15.346914 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8b40c-aa08-46be-b317-9c2c215800a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:15.347515 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8b40c-aa08-46be-b317-9c2c215800a5 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.347792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8b40c-aa08-46be-b317-9c2c215800a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.348688 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8b40c-aa08-46be-b317-9c2c215800a5 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.349315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8b40c-aa08-46be-b317-9c2c215800a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:15.351276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8b40c-aa08-46be-b317-9c2c215800a5 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:15.372068 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1334/2651] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:15 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:15.376177 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fde53d-546c-457e-811c-3b0c251f6a78 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:15.386881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fde53d-546c-457e-811c-3b0c251f6a78 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:15.391501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358bf7d-87dd-469f-a3c4-6ac4d0ae0e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:15.392095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358bf7d-87dd-469f-a3c4-6ac4d0ae0e0f admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.392375 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358bf7d-87dd-469f-a3c4-6ac4d0ae0e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.392648 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358bf7d-87dd-469f-a3c4-6ac4d0ae0e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.393287 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358bf7d-87dd-469f-a3c4-6ac4d0ae0e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:15.395411 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358bf7d-87dd-469f-a3c4-6ac4d0ae0e0f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:15.405155 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1318/2652] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:15.409183 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8b40c-aa08-46be-b317-9c2c215800a5 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:15.427212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8b40c-aa08-46be-b317-9c2c215800a5 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:15.427212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fb548-96c9-4ed5-94f4-c44231e79fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:15.427212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fb548-96c9-4ed5-94f4-c44231e79fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.427212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fb548-96c9-4ed5-94f4-c44231e79fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.427745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fb548-96c9-4ed5-94f4-c44231e79fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.427745 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fb548-96c9-4ed5-94f4-c44231e79fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:15.428902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fb548-96c9-4ed5-94f4-c44231e79fe1 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:15.440067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1335/2653] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:15.444288 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358bf7d-87dd-469f-a3c4-6ac4d0ae0e0f admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:15.454482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358bf7d-87dd-469f-a3c4-6ac4d0ae0e0f admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:15.459831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b54d5d-2b8b-4ab7-bcfd-cfd079f6a980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:15.460481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b54d5d-2b8b-4ab7-bcfd-cfd079f6a980 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.460759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b54d5d-2b8b-4ab7-bcfd-cfd079f6a980 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.461030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b54d5d-2b8b-4ab7-bcfd-cfd079f6a980 admin admin] PATH_INFO: `/v3/projects/020bc8b234aa450c84680a12be34e88c/users/aee83c3328974d47a740d538d9cab401/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.476211 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b54d5d-2b8b-4ab7-bcfd-cfd079f6a980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=020bc8b234aa450c84680a12be34e88c, user_id=aee83c3328974d47a740d538d9cab401, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:15.478952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b54d5d-2b8b-4ab7-bcfd-cfd079f6a980 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:15.495258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1319/2654] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:15 2022] PUT /identity/v3/projects/020bc8b234aa450c84680a12be34e88c/users/aee83c3328974d47a740d538d9cab401/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:15.505560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0411c1-5482-4aab-83d2-ed7027f0f4b3 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.505891 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0411c1-5482-4aab-83d2-ed7027f0f4b3 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.506175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0411c1-5482-4aab-83d2-ed7027f0f4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.536633 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-1f0411c1-5482-4aab-83d2-ed7027f0f4b3 None None] MFA Rules not processed for user `aee83c3328974d47a740d538d9cab401`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:15.537962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-1f0411c1-5482-4aab-83d2-ed7027f0f4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:15.569112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:15.608403 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1336/2655] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:15.612873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b54d5d-2b8b-4ab7-bcfd-cfd079f6a980 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:15.685036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b54d5d-2b8b-4ab7-bcfd-cfd079f6a980 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:15.690924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51598904-7e3e-4f9c-91d3-6dfb3191b041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:15.692263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598904-7e3e-4f9c-91d3-6dfb3191b041 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.692739 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598904-7e3e-4f9c-91d3-6dfb3191b041 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.693108 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51598904-7e3e-4f9c-91d3-6dfb3191b041 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.693935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51598904-7e3e-4f9c-91d3-6dfb3191b041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:15.696104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51598904-7e3e-4f9c-91d3-6dfb3191b041 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:15.725568 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1320/2656] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:15 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:15.729493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0411c1-5482-4aab-83d2-ed7027f0f4b3 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:15.741001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0411c1-5482-4aab-83d2-ed7027f0f4b3 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:15.799096 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa614e-a6e0-497e-97d8-363cda11c2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:15.799711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa614e-a6e0-497e-97d8-363cda11c2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.799994 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa614e-a6e0-497e-97d8-363cda11c2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.800267 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fa614e-a6e0-497e-97d8-363cda11c2c8 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.800931 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa614e-a6e0-497e-97d8-363cda11c2c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:15.802934 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fa614e-a6e0-497e-97d8-363cda11c2c8 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:15.825993 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1337/2657] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:15 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:57:15.829903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51598904-7e3e-4f9c-91d3-6dfb3191b041 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:15.841418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51598904-7e3e-4f9c-91d3-6dfb3191b041 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:15.846115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaef825-0ad3-4775-950d-50b02c28a9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:15.846721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaef825-0ad3-4775-950d-50b02c28a9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.847003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaef825-0ad3-4775-950d-50b02c28a9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.847276 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaef825-0ad3-4775-950d-50b02c28a9a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.847937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaef825-0ad3-4775-950d-50b02c28a9a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:15.849681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaef825-0ad3-4775-950d-50b02c28a9a9 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:15.860651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1321/2658] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:15.864694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa614e-a6e0-497e-97d8-363cda11c2c8 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:15.875264 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fa614e-a6e0-497e-97d8-363cda11c2c8 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:15.880854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebabb9b-cb44-4993-a630-2e52e6b0908a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:15.881502 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebabb9b-cb44-4993-a630-2e52e6b0908a admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.881814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebabb9b-cb44-4993-a630-2e52e6b0908a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.882088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebabb9b-cb44-4993-a630-2e52e6b0908a admin admin] PATH_INFO: `/v3/projects/fa1f5c9110384d6b965879073d4d8d4f/users/c327b39d23444ef988feda1ab017dd68/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.897358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebabb9b-cb44-4993-a630-2e52e6b0908a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1f5c9110384d6b965879073d4d8d4f, user_id=c327b39d23444ef988feda1ab017dd68, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:15.900176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebabb9b-cb44-4993-a630-2e52e6b0908a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:15.913578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1338/2659] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:15 2022] PUT /identity/v3/projects/fa1f5c9110384d6b965879073d4d8d4f/users/c327b39d23444ef988feda1ab017dd68/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:57:15.917515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaef825-0ad3-4775-950d-50b02c28a9a9 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:15.927625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaef825-0ad3-4775-950d-50b02c28a9a9 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:15.947043 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:15.989014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829415e-a117-43b2-8a7c-04956b82ab61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:15.989548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829415e-a117-43b2-8a7c-04956b82ab61 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:15.989856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829415e-a117-43b2-8a7c-04956b82ab61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:15.990130 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829415e-a117-43b2-8a7c-04956b82ab61 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:15.990970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829415e-a117-43b2-8a7c-04956b82ab61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:15.992551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829415e-a117-43b2-8a7c-04956b82ab61 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:16.002522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1322/2660] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:16.007253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebabb9b-cb44-4993-a630-2e52e6b0908a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:16.017356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebabb9b-cb44-4993-a630-2e52e6b0908a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:16.022509 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7ebb8-167a-4a8f-9331-79aed3ca674f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:16.023158 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7ebb8-167a-4a8f-9331-79aed3ca674f admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:16.023436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7ebb8-167a-4a8f-9331-79aed3ca674f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:16.023709 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7ebb8-167a-4a8f-9331-79aed3ca674f admin admin] PATH_INFO: `/v3/projects/fa1f5c9110384d6b965879073d4d8d4f/users/c327b39d23444ef988feda1ab017dd68/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:16.025589 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7ebb8-167a-4a8f-9331-79aed3ca674f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1f5c9110384d6b965879073d4d8d4f, user_id=c327b39d23444ef988feda1ab017dd68, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:16.028243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7ebb8-167a-4a8f-9331-79aed3ca674f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:16.039034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1339/2661] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:16 2022] PUT /identity/v3/projects/fa1f5c9110384d6b965879073d4d8d4f/users/c327b39d23444ef988feda1ab017dd68/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:16.050537 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b4de8-9d38-4b87-b538-95aec37a173f None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:16.050811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b4de8-9d38-4b87-b538-95aec37a173f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:16.051128 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b4de8-9d38-4b87-b538-95aec37a173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:16.202836 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-bd7b4de8-9d38-4b87-b538-95aec37a173f None None] MFA Rules not processed for user `c327b39d23444ef988feda1ab017dd68`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:16.204112 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-bd7b4de8-9d38-4b87-b538-95aec37a173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:16.275937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1323/2662] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:16 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:16.288280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7ebb8-167a-4a8f-9331-79aed3ca674f admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:16.289981 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ffc7ebb8-167a-4a8f-9331-79aed3ca674f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:16.346311 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7ebb8-167a-4a8f-9331-79aed3ca674f admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:16.349856 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a111ba-ffeb-4078-98cd-b003c2b083b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:16.350450 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a111ba-ffeb-4078-98cd-b003c2b083b0 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:16.350725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a111ba-ffeb-4078-98cd-b003c2b083b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:16.350995 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a111ba-ffeb-4078-98cd-b003c2b083b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:16.359634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a111ba-ffeb-4078-98cd-b003c2b083b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:16.361058 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a111ba-ffeb-4078-98cd-b003c2b083b0 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:16.370765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1340/2663] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:16 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:16.677214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b4de8-9d38-4b87-b538-95aec37a173f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:16.686177 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b4de8-9d38-4b87-b538-95aec37a173f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:16.690783 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04d75d-1a29-4748-b553-b81c69cea9df tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c327b39d23444ef988feda1ab017dd68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa1f5c9110384d6b965879073d4d8d4f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:16.691375 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04d75d-1a29-4748-b553-b81c69cea9df tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:16.691658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04d75d-1a29-4748-b553-b81c69cea9df tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:16.691934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04d75d-1a29-4748-b553-b81c69cea9df tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:16.692572 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04d75d-1a29-4748-b553-b81c69cea9df tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:57:16.692882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04d75d-1a29-4748-b553-b81c69cea9df tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:16.694507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04d75d-1a29-4748-b553-b81c69cea9df tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:16.705625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1324/2664] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:57:16.709940 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a111ba-ffeb-4078-98cd-b003c2b083b0 service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:16.719135 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a111ba-ffeb-4078-98cd-b003c2b083b0 service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:16.724552 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79488e4c-d066-44ea-814d-35c2589c82f9 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c327b39d23444ef988feda1ab017dd68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa1f5c9110384d6b965879073d4d8d4f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:16.725156 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79488e4c-d066-44ea-814d-35c2589c82f9 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:16.725441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79488e4c-d066-44ea-814d-35c2589c82f9 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:16.725747 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79488e4c-d066-44ea-814d-35c2589c82f9 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:16.726381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79488e4c-d066-44ea-814d-35c2589c82f9 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:16.728295 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79488e4c-d066-44ea-814d-35c2589c82f9 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:16.756630 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1341/2665] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:16 2022] POST /identity/v3/projects => generated 355 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:16.760733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04d75d-1a29-4748-b553-b81c69cea9df tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:16.770433 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04d75d-1a29-4748-b553-b81c69cea9df tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:16.824669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626a6b8-aea4-4717-8cfe-7b7f78891f07 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c327b39d23444ef988feda1ab017dd68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa1f5c9110384d6b965879073d4d8d4f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:16.825191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626a6b8-aea4-4717-8cfe-7b7f78891f07 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:16.825472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626a6b8-aea4-4717-8cfe-7b7f78891f07 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:16.825773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626a6b8-aea4-4717-8cfe-7b7f78891f07 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:16.826407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626a6b8-aea4-4717-8cfe-7b7f78891f07 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:57:16.826711 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626a6b8-aea4-4717-8cfe-7b7f78891f07 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:16.828293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626a6b8-aea4-4717-8cfe-7b7f78891f07 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:16.839907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1325/2666] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:57:16.876731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79488e4c-d066-44ea-814d-35c2589c82f9 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:16.883273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:16.887178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79488e4c-d066-44ea-814d-35c2589c82f9 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:16.892709 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c327b39d23444ef988feda1ab017dd68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa1f5c9110384d6b965879073d4d8d4f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:16.893315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:16.893602 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:16.893907 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] PATH_INFO: `/v3/projects/fbc5788d09784d86bf21997e957a85f7` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:16.895554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fbc5788d09784d86bf21997e957a85f7)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:16.897461 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:16.944454 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc5788d09784d86bf21997e957a85f7', 'request_id': 'req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:16.951019 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc5788d09784d86bf21997e957a85f7', 'request_id': 'req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:16.957144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc5788d09784d86bf21997e957a85f7', 'request_id': 'req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:16.969572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] The token cache is being invalidate because project fbc5788d09784d86bf21997e957a85f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:16.969889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:16.970519 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-11a1fe70-be68-40b1-80ed-5a6f7a9980a5 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:16.971971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1342/2667] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:16 2022] DELETE /identity/v3/projects/fbc5788d09784d86bf21997e957a85f7 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:17.161210 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626a6b8-aea4-4717-8cfe-7b7f78891f07 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:17.163052 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b626a6b8-aea4-4717-8cfe-7b7f78891f07 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:17.190146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=100 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:17.230119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626a6b8-aea4-4717-8cfe-7b7f78891f07 tempest-VolumeQuotaClassesTest-1835235194 tempest-VolumeQuotaClassesTest-1835235194-project-admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:17.290769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:17.291415 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:17.291740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:17.292061 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] PATH_INFO: `/v3/users/aee83c3328974d47a740d538d9cab401` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:17.293294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=aee83c3328974d47a740d538d9cab401)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:17.295498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:17.352926 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aee83c3328974d47a740d538d9cab401'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:17.353667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aee83c3328974d47a740d538d9cab401'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:17.367138 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aee83c3328974d47a740d538d9cab401'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:17.374029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aee83c3328974d47a740d538d9cab401'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:17.374821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aee83c3328974d47a740d538d9cab401'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:17.391656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1326/2668] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:17 2022] DELETE /identity/v3/users/aee83c3328974d47a740d538d9cab401 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:17.396005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fdb48-54b4-40b6-b31f-1bccd1671824 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:17.396367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fdb48-54b4-40b6-b31f-1bccd1671824 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:17.396711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fdb48-54b4-40b6-b31f-1bccd1671824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:17.416134 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-095fdb48-54b4-40b6-b31f-1bccd1671824 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:17.417425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-095fdb48-54b4-40b6-b31f-1bccd1671824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:17.482075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1343/2669] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:17.492799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:17.494112 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:17.549234 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d7454-bf46-4050-b7c4-fea000f353ea admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:17.553582 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3110b0-2bf9-436b-a38a-0e6121761975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:17.554203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3110b0-2bf9-436b-a38a-0e6121761975 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:17.554477 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3110b0-2bf9-436b-a38a-0e6121761975 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:17.554745 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3110b0-2bf9-436b-a38a-0e6121761975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:17.560473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:17.565326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3110b0-2bf9-436b-a38a-0e6121761975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:17.567613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3110b0-2bf9-436b-a38a-0e6121761975 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:17.577351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1327/2670] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:57:17.848920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fdb48-54b4-40b6-b31f-1bccd1671824 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:17.850413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-095fdb48-54b4-40b6-b31f-1bccd1671824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:17.864081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fdb48-54b4-40b6-b31f-1bccd1671824 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:17.868658 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:17.869358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:17.869731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:17.870072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] PATH_INFO: `/v3/projects/020bc8b234aa450c84680a12be34e88c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:17.871270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=020bc8b234aa450c84680a12be34e88c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:17.874027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:17.922375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '020bc8b234aa450c84680a12be34e88c', 'request_id': 'req-8af7b2b5-4d15-480b-8bb5-614df47a31c6'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:17.928660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '020bc8b234aa450c84680a12be34e88c', 'request_id': 'req-8af7b2b5-4d15-480b-8bb5-614df47a31c6'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:17.934471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '020bc8b234aa450c84680a12be34e88c', 'request_id': 'req-8af7b2b5-4d15-480b-8bb5-614df47a31c6'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:17.947122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] The token cache is being invalidate because project 020bc8b234aa450c84680a12be34e88c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:17.947414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:17.948045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:17.950308 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1344/2671] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:17 2022] DELETE /identity/v3/projects/020bc8b234aa450c84680a12be34e88c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:57:17.955170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3110b0-2bf9-436b-a38a-0e6121761975 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:17.965994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3110b0-2bf9-436b-a38a-0e6121761975 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:18.024343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea5b7d-d423-4808-8037-38c02b7d72b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:18.024967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea5b7d-d423-4808-8037-38c02b7d72b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:18.025245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea5b7d-d423-4808-8037-38c02b7d72b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:18.025514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea5b7d-d423-4808-8037-38c02b7d72b8 admin admin] PATH_INFO: `/v3/users/c327b39d23444ef988feda1ab017dd68` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:18.026656 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea5b7d-d423-4808-8037-38c02b7d72b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c327b39d23444ef988feda1ab017dd68)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:18.028616 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea5b7d-d423-4808-8037-38c02b7d72b8 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:18.074762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c8ea5b7d-d423-4808-8037-38c02b7d72b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c327b39d23444ef988feda1ab017dd68'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:18.075414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c8ea5b7d-d423-4808-8037-38c02b7d72b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c327b39d23444ef988feda1ab017dd68'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:18.087679 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c8ea5b7d-d423-4808-8037-38c02b7d72b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c327b39d23444ef988feda1ab017dd68'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:18.094464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c8ea5b7d-d423-4808-8037-38c02b7d72b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c327b39d23444ef988feda1ab017dd68'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:18.095129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c8ea5b7d-d423-4808-8037-38c02b7d72b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c327b39d23444ef988feda1ab017dd68'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:18.107480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1328/2672] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:17 2022] DELETE /identity/v3/users/c327b39d23444ef988feda1ab017dd68 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:18.367315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:18.368898 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:18.382690 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7b2b5-4d15-480b-8bb5-614df47a31c6 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:18.402917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:18.458386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:18.458943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:18.459225 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:18.459501 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] PATH_INFO: `/v3/projects/fa1f5c9110384d6b965879073d4d8d4f` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:18.460600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa1f5c9110384d6b965879073d4d8d4f)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:18.462539 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:18.508929 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa1f5c9110384d6b965879073d4d8d4f', 'request_id': 'req-a493b0b9-db8b-49da-9f9a-2743c01d2c16'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:18.515816 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa1f5c9110384d6b965879073d4d8d4f', 'request_id': 'req-a493b0b9-db8b-49da-9f9a-2743c01d2c16'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:18.521910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa1f5c9110384d6b965879073d4d8d4f', 'request_id': 'req-a493b0b9-db8b-49da-9f9a-2743c01d2c16'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:18.535160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] The token cache is being invalidate because project fa1f5c9110384d6b965879073d4d8d4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:18.535370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:18.536078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:18.537542 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1345/2673] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:18 2022] DELETE /identity/v3/projects/fa1f5c9110384d6b965879073d4d8d4f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:18.542561 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65f691-3b5a-4fdd-b5ba-d9507c7b2a3c None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:18.543068 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65f691-3b5a-4fdd-b5ba-d9507c7b2a3c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:18.543196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65f691-3b5a-4fdd-b5ba-d9507c7b2a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:18.568050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-1a65f691-3b5a-4fdd-b5ba-d9507c7b2a3c None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:18.569419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1a65f691-3b5a-4fdd-b5ba-d9507c7b2a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:18.598468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:18.599879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:18.637370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1329/2674] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:18.645113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cbace-ca64-4a1f-b177-d29851fa4fdd None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:18.645434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cbace-ca64-4a1f-b177-d29851fa4fdd None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:18.645914 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cbace-ca64-4a1f-b177-d29851fa4fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:18.663200 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493b0b9-db8b-49da-9f9a-2743c01d2c16 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:18.665214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-6e4cbace-ca64-4a1f-b177-d29851fa4fdd None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:18.666447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6e4cbace-ca64-4a1f-b177-d29851fa4fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:18.667174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36155063-093e-4375-b871-ee703395e5bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:18.667806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36155063-093e-4375-b871-ee703395e5bf service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:18.668083 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36155063-093e-4375-b871-ee703395e5bf service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:18.668352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36155063-093e-4375-b871-ee703395e5bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:18.670104 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-36155063-093e-4375-b871-ee703395e5bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:18.674385 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1330/2675] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:18.679056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cbace-ca64-4a1f-b177-d29851fa4fdd None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:18.683340 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36155063-093e-4375-b871-ee703395e5bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:18.684776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36155063-093e-4375-b871-ee703395e5bf service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:18.688339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cbace-ca64-4a1f-b177-d29851fa4fdd None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:18.694164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ca10c-3ebf-416e-a0eb-f6ef3245a897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:18.694877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ca10c-3ebf-416e-a0eb-f6ef3245a897 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:18.695152 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ca10c-3ebf-416e-a0eb-f6ef3245a897 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:18.695424 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ca10c-3ebf-416e-a0eb-f6ef3245a897 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:18.695947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1346/2676] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:57:18.696066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ca10c-3ebf-416e-a0eb-f6ef3245a897 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:57:18.696341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ca10c-3ebf-416e-a0eb-f6ef3245a897 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:18.698030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ca10c-3ebf-416e-a0eb-f6ef3245a897 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:18.699634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:18.713848 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1331/2677] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 08:57:18.716468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36155063-093e-4375-b871-ee703395e5bf service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:18.727036 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36155063-093e-4375-b871-ee703395e5bf service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:18.732699 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e10f6d-d9ca-4b16-ac65-173b67242e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:18.733314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e10f6d-d9ca-4b16-ac65-173b67242e2f admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:18.733598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e10f6d-d9ca-4b16-ac65-173b67242e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:18.733917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e10f6d-d9ca-4b16-ac65-173b67242e2f admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:18.734601 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e10f6d-d9ca-4b16-ac65-173b67242e2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:18.736976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e10f6d-d9ca-4b16-ac65-173b67242e2f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:18.765861 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1347/2678] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:18 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:18.770162 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ca10c-3ebf-416e-a0eb-f6ef3245a897 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:18.780146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ca10c-3ebf-416e-a0eb-f6ef3245a897 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:18.836504 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f19ee-898c-4478-b099-4e7f2d70bc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:18.837102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f19ee-898c-4478-b099-4e7f2d70bc80 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:18.837383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f19ee-898c-4478-b099-4e7f2d70bc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:18.837650 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f19ee-898c-4478-b099-4e7f2d70bc80 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:18.838294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f19ee-898c-4478-b099-4e7f2d70bc80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:18.840190 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83f19ee-898c-4478-b099-4e7f2d70bc80 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:18.863531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1332/2679] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:18 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:18.865271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e10f6d-d9ca-4b16-ac65-173b67242e2f admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:18.875532 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e10f6d-d9ca-4b16-ac65-173b67242e2f admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:18.880151 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9f1be-edd8-4ad2-8685-921ced13d39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:18.880751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9f1be-edd8-4ad2-8685-921ced13d39d admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:18.881027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9f1be-edd8-4ad2-8685-921ced13d39d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:18.881298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9f1be-edd8-4ad2-8685-921ced13d39d admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:18.881973 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9f1be-edd8-4ad2-8685-921ced13d39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:18.883582 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9f1be-edd8-4ad2-8685-921ced13d39d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:18.893420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1348/2680] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 08:57:18.897463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f19ee-898c-4478-b099-4e7f2d70bc80 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:18.907218 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f19ee-898c-4478-b099-4e7f2d70bc80 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:18.912318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0471cb-a005-422b-89d8-119eea23efdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:18.912917 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0471cb-a005-422b-89d8-119eea23efdb admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:18.913196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0471cb-a005-422b-89d8-119eea23efdb admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:18.913463 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0471cb-a005-422b-89d8-119eea23efdb admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:18.914139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0471cb-a005-422b-89d8-119eea23efdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:18.915721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0471cb-a005-422b-89d8-119eea23efdb admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:18.926760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1333/2681] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:18.930989 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9f1be-edd8-4ad2-8685-921ced13d39d admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:18.940905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9f1be-edd8-4ad2-8685-921ced13d39d admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:18.945617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2f645-ccd3-41cf-91d6-a285b889c64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:18.946503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2f645-ccd3-41cf-91d6-a285b889c64c admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:18.946851 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2f645-ccd3-41cf-91d6-a285b889c64c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:18.947192 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2f645-ccd3-41cf-91d6-a285b889c64c admin admin] PATH_INFO: `/v3/projects/82e3521f2d9245b48bfbfecccf6b5204/users/2b1e761175f74d02b7800ad74cd188e3/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:18.955780 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:18.963193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2f645-ccd3-41cf-91d6-a285b889c64c admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e3521f2d9245b48bfbfecccf6b5204, user_id=2b1e761175f74d02b7800ad74cd188e3, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:18.966387 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2f645-ccd3-41cf-91d6-a285b889c64c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:18.981799 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1349/2682] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:18 2022] PUT /identity/v3/projects/82e3521f2d9245b48bfbfecccf6b5204/users/2b1e761175f74d02b7800ad74cd188e3/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:18.993640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d14bc-70f8-4fad-98f2-71d0813fb138 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:18.993954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d14bc-70f8-4fad-98f2-71d0813fb138 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:18.994237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d14bc-70f8-4fad-98f2-71d0813fb138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.021329 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-e45d14bc-70f8-4fad-98f2-71d0813fb138 None None] MFA Rules not processed for user `2b1e761175f74d02b7800ad74cd188e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:19.022560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e45d14bc-70f8-4fad-98f2-71d0813fb138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:19.087758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1334/2683] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:19.092190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2f645-ccd3-41cf-91d6-a285b889c64c admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:19.154076 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2f645-ccd3-41cf-91d6-a285b889c64c admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:19.158810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc97e7e-6994-4de3-9504-7ae93ecc4fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:19.159516 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc97e7e-6994-4de3-9504-7ae93ecc4fff admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.159889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc97e7e-6994-4de3-9504-7ae93ecc4fff admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.160259 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc97e7e-6994-4de3-9504-7ae93ecc4fff admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.161647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc97e7e-6994-4de3-9504-7ae93ecc4fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:19.163742 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc97e7e-6994-4de3-9504-7ae93ecc4fff admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:19.194769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1350/2684] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:19 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:19.196179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d14bc-70f8-4fad-98f2-71d0813fb138 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:19.206565 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d14bc-70f8-4fad-98f2-71d0813fb138 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:19.270753 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aef005-07e7-4482-af57-9f7ba865b9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:19.271469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aef005-07e7-4482-af57-9f7ba865b9d5 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.271745 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aef005-07e7-4482-af57-9f7ba865b9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.272012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aef005-07e7-4482-af57-9f7ba865b9d5 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.272642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aef005-07e7-4482-af57-9f7ba865b9d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:19.274576 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aef005-07e7-4482-af57-9f7ba865b9d5 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:19.283509 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:19.297335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1335/2685] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:19 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:19.301376 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc97e7e-6994-4de3-9504-7ae93ecc4fff admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:19.311910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc97e7e-6994-4de3-9504-7ae93ecc4fff admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:19.316628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38131e6-d599-4ea0-be3e-aaca1c60077b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:19.317274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38131e6-d599-4ea0-be3e-aaca1c60077b admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.317587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38131e6-d599-4ea0-be3e-aaca1c60077b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.317917 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38131e6-d599-4ea0-be3e-aaca1c60077b admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.318595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38131e6-d599-4ea0-be3e-aaca1c60077b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:19.320298 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38131e6-d599-4ea0-be3e-aaca1c60077b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:19.331390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1351/2686] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:19.335905 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aef005-07e7-4482-af57-9f7ba865b9d5 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:19.346360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aef005-07e7-4482-af57-9f7ba865b9d5 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:19.351945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58d76f-afbd-45a0-b8ec-7a0d450ae025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:19.352790 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58d76f-afbd-45a0-b8ec-7a0d450ae025 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.352954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58d76f-afbd-45a0-b8ec-7a0d450ae025 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.353306 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58d76f-afbd-45a0-b8ec-7a0d450ae025 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.354084 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58d76f-afbd-45a0-b8ec-7a0d450ae025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:19.355767 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58d76f-afbd-45a0-b8ec-7a0d450ae025 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:19.365833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1336/2687] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:19.370132 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38131e6-d599-4ea0-be3e-aaca1c60077b admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:19.380265 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38131e6-d599-4ea0-be3e-aaca1c60077b admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:19.385481 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad0825-8def-48f8-a4e2-803df5639cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:19.386399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad0825-8def-48f8-a4e2-803df5639cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.386589 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad0825-8def-48f8-a4e2-803df5639cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.386853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad0825-8def-48f8-a4e2-803df5639cf4 admin admin] PATH_INFO: `/v3/projects/d25b926ee1a943e08a1a9c36d835e6cd/users/0421362219be489e879b7c8988e4a2c9/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.402377 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad0825-8def-48f8-a4e2-803df5639cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25b926ee1a943e08a1a9c36d835e6cd, user_id=0421362219be489e879b7c8988e4a2c9, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:19.405088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad0825-8def-48f8-a4e2-803df5639cf4 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:19.418737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1352/2688] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:19 2022] PUT /identity/v3/projects/d25b926ee1a943e08a1a9c36d835e6cd/users/0421362219be489e879b7c8988e4a2c9/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:19.429131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaaa10-834f-4ff9-bbe7-8c2070fd9608 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.429418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaaa10-834f-4ff9-bbe7-8c2070fd9608 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.429710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfaaa10-834f-4ff9-bbe7-8c2070fd9608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.458612 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-edfaaa10-834f-4ff9-bbe7-8c2070fd9608 None None] MFA Rules not processed for user `0421362219be489e879b7c8988e4a2c9`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:19.459814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-edfaaa10-834f-4ff9-bbe7-8c2070fd9608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:19.524861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1337/2689] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:57:19.529110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad0825-8def-48f8-a4e2-803df5639cf4 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:19.598905 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad0825-8def-48f8-a4e2-803df5639cf4 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:19.604191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b98028-d0c4-4dc0-91ad-f857d1d343b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:19.605080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b98028-d0c4-4dc0-91ad-f857d1d343b8 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.605586 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b98028-d0c4-4dc0-91ad-f857d1d343b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.605947 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b98028-d0c4-4dc0-91ad-f857d1d343b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.606607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b98028-d0c4-4dc0-91ad-f857d1d343b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:19.608572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b98028-d0c4-4dc0-91ad-f857d1d343b8 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:19.640580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1353/2690] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:19 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:19.644940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfaaa10-834f-4ff9-bbe7-8c2070fd9608 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:19.658135 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfaaa10-834f-4ff9-bbe7-8c2070fd9608 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:19.731789 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4443d7e-2f14-446e-be81-eade06e77dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:19.732284 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4443d7e-2f14-446e-be81-eade06e77dda admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.732645 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4443d7e-2f14-446e-be81-eade06e77dda admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.733462 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4443d7e-2f14-446e-be81-eade06e77dda admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.734025 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4443d7e-2f14-446e-be81-eade06e77dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:19.735968 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4443d7e-2f14-446e-be81-eade06e77dda admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:19.760148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1338/2691] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:19 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:19.764149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b98028-d0c4-4dc0-91ad-f857d1d343b8 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:19.775284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b98028-d0c4-4dc0-91ad-f857d1d343b8 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:19.779857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fdc4c-f39c-480c-b666-879ad381e01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:19.780457 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fdc4c-f39c-480c-b666-879ad381e01c admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.780731 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fdc4c-f39c-480c-b666-879ad381e01c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.781001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0fdc4c-f39c-480c-b666-879ad381e01c admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.781667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fdc4c-f39c-480c-b666-879ad381e01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:19.783369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0fdc4c-f39c-480c-b666-879ad381e01c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:19.794500 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1354/2692] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:19.798821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4443d7e-2f14-446e-be81-eade06e77dda admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:19.809445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4443d7e-2f14-446e-be81-eade06e77dda admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:19.814074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb172f-498f-4e90-9ed8-829c38797bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:19.814782 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb172f-498f-4e90-9ed8-829c38797bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.815016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb172f-498f-4e90-9ed8-829c38797bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.815291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb172f-498f-4e90-9ed8-829c38797bf2 admin admin] PATH_INFO: `/v3/projects/693962861175449387d03608387d5fd2/users/07c61abec08d4d258a0704cd15de8561/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.831510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb172f-498f-4e90-9ed8-829c38797bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=693962861175449387d03608387d5fd2, user_id=07c61abec08d4d258a0704cd15de8561, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:19.834953 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb172f-498f-4e90-9ed8-829c38797bf2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:19.849363 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1339/2693] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:19 2022] PUT /identity/v3/projects/693962861175449387d03608387d5fd2/users/07c61abec08d4d258a0704cd15de8561/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:19.853607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fdc4c-f39c-480c-b666-879ad381e01c admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:19.863737 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0fdc4c-f39c-480c-b666-879ad381e01c admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:19.872599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:19.933786 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8fb2c-fde9-472d-9370-53f959a995d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:19.934405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8fb2c-fde9-472d-9370-53f959a995d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.934685 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8fb2c-fde9-472d-9370-53f959a995d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.934963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8fb2c-fde9-472d-9370-53f959a995d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.935599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8fb2c-fde9-472d-9370-53f959a995d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:19.937265 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8fb2c-fde9-472d-9370-53f959a995d2 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:19.947389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1355/2694] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:19.951344 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb172f-498f-4e90-9ed8-829c38797bf2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:19.962703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb172f-498f-4e90-9ed8-829c38797bf2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:19.967513 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5169a1-74cb-434a-9af1-3d9430c0e5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:19.968161 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5169a1-74cb-434a-9af1-3d9430c0e5a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.968440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5169a1-74cb-434a-9af1-3d9430c0e5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.968726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5169a1-74cb-434a-9af1-3d9430c0e5a2 admin admin] PATH_INFO: `/v3/projects/693962861175449387d03608387d5fd2/users/07c61abec08d4d258a0704cd15de8561/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:19.970628 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5169a1-74cb-434a-9af1-3d9430c0e5a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=693962861175449387d03608387d5fd2, user_id=07c61abec08d4d258a0704cd15de8561, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:19.974117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5169a1-74cb-434a-9af1-3d9430c0e5a2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:19.984195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1340/2695] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:19 2022] PUT /identity/v3/projects/693962861175449387d03608387d5fd2/users/07c61abec08d4d258a0704cd15de8561/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:19.994260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9f156-0089-4960-9e88-7eb992a5bc3b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:19.994548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9f156-0089-4960-9e88-7eb992a5bc3b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:19.994822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9f156-0089-4960-9e88-7eb992a5bc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:20.024406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-21d9f156-0089-4960-9e88-7eb992a5bc3b None None] MFA Rules not processed for user `07c61abec08d4d258a0704cd15de8561`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:20.025724 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-21d9f156-0089-4960-9e88-7eb992a5bc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:20.109573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1356/2696] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:20.120670 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5169a1-74cb-434a-9af1-3d9430c0e5a2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:20.122447 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ea5169a1-74cb-434a-9af1-3d9430c0e5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:20.157877 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:20.198459 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5169a1-74cb-434a-9af1-3d9430c0e5a2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:20.202745 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e226e1-ce40-4c36-b768-2a3a4e334e48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:20.203736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e226e1-ce40-4c36-b768-2a3a4e334e48 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:20.203736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e226e1-ce40-4c36-b768-2a3a4e334e48 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:20.203973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e226e1-ce40-4c36-b768-2a3a4e334e48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:20.216426 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e226e1-ce40-4c36-b768-2a3a4e334e48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:20.217914 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e226e1-ce40-4c36-b768-2a3a4e334e48 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:20.228203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1341/2697] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:20.808788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9f156-0089-4960-9e88-7eb992a5bc3b None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:20.810274 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-21d9f156-0089-4960-9e88-7eb992a5bc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:20.822707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9f156-0089-4960-9e88-7eb992a5bc3b None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:20.826248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef017ba-43a3-422e-b059-62c34d476520 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:20.826849 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef017ba-43a3-422e-b059-62c34d476520 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:20.827187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef017ba-43a3-422e-b059-62c34d476520 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:20.827429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef017ba-43a3-422e-b059-62c34d476520 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:20.893396 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef017ba-43a3-422e-b059-62c34d476520 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:20.895248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef017ba-43a3-422e-b059-62c34d476520 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:20.906758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1357/2698] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:22.130675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e226e1-ce40-4c36-b768-2a3a4e334e48 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:22.132136 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-38e226e1-ce40-4c36-b768-2a3a4e334e48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:22.197023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e226e1-ce40-4c36-b768-2a3a4e334e48 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:22.201659 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a659b7-f9af-48b5-98c6-ffacabbfc748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:22.202307 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a659b7-f9af-48b5-98c6-ffacabbfc748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:22.202591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a659b7-f9af-48b5-98c6-ffacabbfc748 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:22.202866 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a659b7-f9af-48b5-98c6-ffacabbfc748 admin admin] PATH_INFO: `/v3/users/ed4b7bfb87b144b38b82514c17a468bc` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:22.204001 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a659b7-f9af-48b5-98c6-ffacabbfc748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4b7bfb87b144b38b82514c17a468bc)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:22.205952 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a659b7-f9af-48b5-98c6-ffacabbfc748 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:22.254898 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c6a659b7-f9af-48b5-98c6-ffacabbfc748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4b7bfb87b144b38b82514c17a468bc'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:22.255569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c6a659b7-f9af-48b5-98c6-ffacabbfc748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4b7bfb87b144b38b82514c17a468bc'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:22.268566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c6a659b7-f9af-48b5-98c6-ffacabbfc748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4b7bfb87b144b38b82514c17a468bc'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:22.275726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c6a659b7-f9af-48b5-98c6-ffacabbfc748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4b7bfb87b144b38b82514c17a468bc'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:22.276417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c6a659b7-f9af-48b5-98c6-ffacabbfc748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4b7bfb87b144b38b82514c17a468bc'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:22.289283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1342/2699] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:22 2022] DELETE /identity/v3/users/ed4b7bfb87b144b38b82514c17a468bc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:22.478779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef017ba-43a3-422e-b059-62c34d476520 service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:22.491370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef017ba-43a3-422e-b059-62c34d476520 service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:22.520066 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:22.550233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67211d7c-b12c-4f17-aff9-1e7620b4b604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:22.550879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67211d7c-b12c-4f17-aff9-1e7620b4b604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:22.551832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67211d7c-b12c-4f17-aff9-1e7620b4b604 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:22.552108 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67211d7c-b12c-4f17-aff9-1e7620b4b604 admin admin] PATH_INFO: `/v3/projects/ed8dda1234f54905962e427bf75e82c1` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:22.553260 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67211d7c-b12c-4f17-aff9-1e7620b4b604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed8dda1234f54905962e427bf75e82c1)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:22.556839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67211d7c-b12c-4f17-aff9-1e7620b4b604 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:22.601756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-67211d7c-b12c-4f17-aff9-1e7620b4b604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed8dda1234f54905962e427bf75e82c1', 'request_id': 'req-67211d7c-b12c-4f17-aff9-1e7620b4b604'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:22.607739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-67211d7c-b12c-4f17-aff9-1e7620b4b604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed8dda1234f54905962e427bf75e82c1', 'request_id': 'req-67211d7c-b12c-4f17-aff9-1e7620b4b604'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:22.613195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-67211d7c-b12c-4f17-aff9-1e7620b4b604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed8dda1234f54905962e427bf75e82c1', 'request_id': 'req-67211d7c-b12c-4f17-aff9-1e7620b4b604'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:22.626246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-67211d7c-b12c-4f17-aff9-1e7620b4b604 admin admin] The token cache is being invalidate because project ed8dda1234f54905962e427bf75e82c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:22.626531 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-67211d7c-b12c-4f17-aff9-1e7620b4b604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:22.627147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-67211d7c-b12c-4f17-aff9-1e7620b4b604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:22.628626 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1358/2700] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:22 2022] DELETE /identity/v3/projects/ed8dda1234f54905962e427bf75e82c1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:22.633726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0f86c-f3ff-48e5-b954-884efbc0a4fa None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:22.634002 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0f86c-f3ff-48e5-b954-884efbc0a4fa None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:22.634276 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0f86c-f3ff-48e5-b954-884efbc0a4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:22.655705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-06f0f86c-f3ff-48e5-b954-884efbc0a4fa None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:22.656735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-06f0f86c-f3ff-48e5-b954-884efbc0a4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:22.698893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:22.721259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1343/2701] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:22.729612 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd8705-13c2-4d09-8208-995a280af3d5 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:22.729936 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd8705-13c2-4d09-8208-995a280af3d5 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:22.730209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd8705-13c2-4d09-8208-995a280af3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:22.751938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-77cd8705-13c2-4d09-8208-995a280af3d5 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:22.753189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-77cd8705-13c2-4d09-8208-995a280af3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:22.761153 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1359/2702] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:22.765432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0f86c-f3ff-48e5-b954-884efbc0a4fa None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:22.775098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0f86c-f3ff-48e5-b954-884efbc0a4fa None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:22.779673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a309828-7552-4905-bde1-401ecb0298a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:22.780272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a309828-7552-4905-bde1-401ecb0298a2 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:22.780556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a309828-7552-4905-bde1-401ecb0298a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:22.780828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a309828-7552-4905-bde1-401ecb0298a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:22.781466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a309828-7552-4905-bde1-401ecb0298a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:57:22.781803 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a309828-7552-4905-bde1-401ecb0298a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:22.783399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a309828-7552-4905-bde1-401ecb0298a2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:22.795174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1344/2703] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:57:22.799293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd8705-13c2-4d09-8208-995a280af3d5 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:22.809122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd8705-13c2-4d09-8208-995a280af3d5 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:22.813646 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174985f-ab90-4297-893e-d85d8bf75b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:22.814272 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174985f-ab90-4297-893e-d85d8bf75b8c admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:22.814550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174985f-ab90-4297-893e-d85d8bf75b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:22.814820 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174985f-ab90-4297-893e-d85d8bf75b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:22.815440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174985f-ab90-4297-893e-d85d8bf75b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:22.817332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174985f-ab90-4297-893e-d85d8bf75b8c admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:22.843961 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1360/2704] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:22 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:22.848915 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a309828-7552-4905-bde1-401ecb0298a2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:22.858772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a309828-7552-4905-bde1-401ecb0298a2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:22.912916 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6e26a-1925-4cbd-ab02-d2be78be1bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:22.913519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6e26a-1925-4cbd-ab02-d2be78be1bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:22.913827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6e26a-1925-4cbd-ab02-d2be78be1bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:22.914103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6e26a-1925-4cbd-ab02-d2be78be1bf6 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:22.914736 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6e26a-1925-4cbd-ab02-d2be78be1bf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:22.916645 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6e26a-1925-4cbd-ab02-d2be78be1bf6 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:22.938255 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1345/2705] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:22 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:22.942391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174985f-ab90-4297-893e-d85d8bf75b8c admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:22.952430 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174985f-ab90-4297-893e-d85d8bf75b8c admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:22.957220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a828e-c869-4233-8bb0-b4513b998d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:22.958350 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a828e-c869-4233-8bb0-b4513b998d52 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:22.958627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a828e-c869-4233-8bb0-b4513b998d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:22.958894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a828e-c869-4233-8bb0-b4513b998d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:22.959524 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a828e-c869-4233-8bb0-b4513b998d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:22.961112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a828e-c869-4233-8bb0-b4513b998d52 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:22.970696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1361/2706] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:22.975029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6e26a-1925-4cbd-ab02-d2be78be1bf6 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:22.984573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6e26a-1925-4cbd-ab02-d2be78be1bf6 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:22.989095 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88154067-0b98-49e0-8a70-9bf2fdb0cf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:22.989674 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88154067-0b98-49e0-8a70-9bf2fdb0cf65 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:22.989988 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88154067-0b98-49e0-8a70-9bf2fdb0cf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:22.990261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88154067-0b98-49e0-8a70-9bf2fdb0cf65 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:22.990890 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88154067-0b98-49e0-8a70-9bf2fdb0cf65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:22.992474 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88154067-0b98-49e0-8a70-9bf2fdb0cf65 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:23.002668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1346/2707] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:23.006459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a828e-c869-4233-8bb0-b4513b998d52 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:23.016091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a828e-c869-4233-8bb0-b4513b998d52 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:23.021219 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9981c4f-7fbb-4330-bdac-b33f34abb8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:23.021888 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9981c4f-7fbb-4330-bdac-b33f34abb8a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:23.022165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9981c4f-7fbb-4330-bdac-b33f34abb8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:23.022436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9981c4f-7fbb-4330-bdac-b33f34abb8a7 admin admin] PATH_INFO: `/v3/projects/4370b750fe184c5eade1378abfa02752/users/c710350e02634e84b3d4700ae5fa6925/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:23.037303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9981c4f-7fbb-4330-bdac-b33f34abb8a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4370b750fe184c5eade1378abfa02752, user_id=c710350e02634e84b3d4700ae5fa6925, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:23.039961 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9981c4f-7fbb-4330-bdac-b33f34abb8a7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:23.052253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1362/2708] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:23 2022] PUT /identity/v3/projects/4370b750fe184c5eade1378abfa02752/users/c710350e02634e84b3d4700ae5fa6925/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:23.057100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d5d8-8577-4ee2-ae76-6b3a1ef2725d None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:23.057390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d5d8-8577-4ee2-ae76-6b3a1ef2725d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:23.057661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d5d8-8577-4ee2-ae76-6b3a1ef2725d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:23.075887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d2c8d5d8-8577-4ee2-ae76-6b3a1ef2725d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:23.077062 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d2c8d5d8-8577-4ee2-ae76-6b3a1ef2725d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:23.132826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1347/2709] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:23.142322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9981c4f-7fbb-4330-bdac-b33f34abb8a7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:23.144041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a9981c4f-7fbb-4330-bdac-b33f34abb8a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:23.200753 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9981c4f-7fbb-4330-bdac-b33f34abb8a7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:23.204502 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491165a-686d-435f-96ef-ae3fc2d828a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:23.205104 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491165a-686d-435f-96ef-ae3fc2d828a3 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:23.205386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491165a-686d-435f-96ef-ae3fc2d828a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:23.205662 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491165a-686d-435f-96ef-ae3fc2d828a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:23.215545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491165a-686d-435f-96ef-ae3fc2d828a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:23.217152 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491165a-686d-435f-96ef-ae3fc2d828a3 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:23.222098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:23.227198 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1363/2710] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:25.038124 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8d5d8-8577-4ee2-ae76-6b3a1ef2725d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:25.039603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d2c8d5d8-8577-4ee2-ae76-6b3a1ef2725d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:25.098170 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8d5d8-8577-4ee2-ae76-6b3a1ef2725d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:25.102441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241a09a-2e2f-48ad-8fe7-218cecc1a6a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:25.102886 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241a09a-2e2f-48ad-8fe7-218cecc1a6a1 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:25.103279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241a09a-2e2f-48ad-8fe7-218cecc1a6a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:25.103651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241a09a-2e2f-48ad-8fe7-218cecc1a6a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:25.105934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5241a09a-2e2f-48ad-8fe7-218cecc1a6a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:25.111458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:25.175969 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241a09a-2e2f-48ad-8fe7-218cecc1a6a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:25.177986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241a09a-2e2f-48ad-8fe7-218cecc1a6a1 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:25.188987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1348/2711] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:27.044778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2f379-051b-45d0-9c56-c0d8ac076a46 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:27.045255 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2f379-051b-45d0-9c56-c0d8ac076a46 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:27.045379 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b2f379-051b-45d0-9c56-c0d8ac076a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:27.078584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-c4b2f379-051b-45d0-9c56-c0d8ac076a46 None None] MFA Rules not processed for user `c710350e02634e84b3d4700ae5fa6925`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:27.079514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c4b2f379-051b-45d0-9c56-c0d8ac076a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:27.175648 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1364/2712] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:27.191595 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241a09a-2e2f-48ad-8fe7-218cecc1a6a1 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:27.196140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5241a09a-2e2f-48ad-8fe7-218cecc1a6a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:27.307821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241a09a-2e2f-48ad-8fe7-218cecc1a6a1 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:27.313443 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56741282-ed9b-4ecd-b88f-98b882b2dfec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:27.314165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56741282-ed9b-4ecd-b88f-98b882b2dfec service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:27.314445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56741282-ed9b-4ecd-b88f-98b882b2dfec service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:27.314725 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56741282-ed9b-4ecd-b88f-98b882b2dfec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:27.323437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56741282-ed9b-4ecd-b88f-98b882b2dfec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:27.331627 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56741282-ed9b-4ecd-b88f-98b882b2dfec service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:27.349634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1349/2713] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:27.605370 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2f379-051b-45d0-9c56-c0d8ac076a46 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:27.606840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c4b2f379-051b-45d0-9c56-c0d8ac076a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:27.625342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b2f379-051b-45d0-9c56-c0d8ac076a46 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:27.631373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:27.632080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:27.632489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:27.632771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] PATH_INFO: `/v3/users/2b1e761175f74d02b7800ad74cd188e3` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:27.634764 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1e761175f74d02b7800ad74cd188e3)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:27.636893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:27.706908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1e761175f74d02b7800ad74cd188e3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:27.709346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1e761175f74d02b7800ad74cd188e3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:27.718621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1e761175f74d02b7800ad74cd188e3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:27.719992 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1e761175f74d02b7800ad74cd188e3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:27.748478 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1e761175f74d02b7800ad74cd188e3'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:27.764271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1365/2714] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:27 2022] DELETE /identity/v3/users/2b1e761175f74d02b7800ad74cd188e3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:27.768994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6fa3d-e98c-4a98-85df-1111041d4952 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:27.769539 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6fa3d-e98c-4a98-85df-1111041d4952 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:27.770893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6fa3d-e98c-4a98-85df-1111041d4952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:27.800774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-2db6fa3d-e98c-4a98-85df-1111041d4952 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:27.802499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2db6fa3d-e98c-4a98-85df-1111041d4952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:27.877551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1350/2715] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:57:27.889577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:27.891631 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:27.967852 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e007ebc5-80b7-406e-9da8-dd6ca72119e0 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:27.971769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317eaa3e-30d3-458e-8313-79d91242a85d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:27.972450 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317eaa3e-30d3-458e-8313-79d91242a85d service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:27.972806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317eaa3e-30d3-458e-8313-79d91242a85d service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:27.973162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317eaa3e-30d3-458e-8313-79d91242a85d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:27.983361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317eaa3e-30d3-458e-8313-79d91242a85d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:27.984842 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317eaa3e-30d3-458e-8313-79d91242a85d service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:27.995647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1366/2716] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:28.466097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6fa3d-e98c-4a98-85df-1111041d4952 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:28.468228 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2db6fa3d-e98c-4a98-85df-1111041d4952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:28.482159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6fa3d-e98c-4a98-85df-1111041d4952 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:28.498207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906be9f-9073-4f0e-80d7-79011be98e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:28.498873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906be9f-9073-4f0e-80d7-79011be98e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:28.499152 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906be9f-9073-4f0e-80d7-79011be98e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:28.499432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906be9f-9073-4f0e-80d7-79011be98e7c admin admin] PATH_INFO: `/v3/projects/82e3521f2d9245b48bfbfecccf6b5204` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:28.501271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906be9f-9073-4f0e-80d7-79011be98e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=82e3521f2d9245b48bfbfecccf6b5204)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:28.503098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906be9f-9073-4f0e-80d7-79011be98e7c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:28.551718 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:28.573520 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8906be9f-9073-4f0e-80d7-79011be98e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82e3521f2d9245b48bfbfecccf6b5204', 'request_id': 'req-8906be9f-9073-4f0e-80d7-79011be98e7c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:28.583918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8906be9f-9073-4f0e-80d7-79011be98e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82e3521f2d9245b48bfbfecccf6b5204', 'request_id': 'req-8906be9f-9073-4f0e-80d7-79011be98e7c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:28.590771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8906be9f-9073-4f0e-80d7-79011be98e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82e3521f2d9245b48bfbfecccf6b5204', 'request_id': 'req-8906be9f-9073-4f0e-80d7-79011be98e7c'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:28.608410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8906be9f-9073-4f0e-80d7-79011be98e7c admin admin] The token cache is being invalidate because project 82e3521f2d9245b48bfbfecccf6b5204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:28.608708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8906be9f-9073-4f0e-80d7-79011be98e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:28.609533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-8906be9f-9073-4f0e-80d7-79011be98e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:28.611304 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1351/2717] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:28 2022] DELETE /identity/v3/projects/82e3521f2d9245b48bfbfecccf6b5204 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:28.615912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317eaa3e-30d3-458e-8313-79d91242a85d service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:28.623863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:28.630472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317eaa3e-30d3-458e-8313-79d91242a85d service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:28.708713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:28.709773 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:28.710104 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:28.710815 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] PATH_INFO: `/v3/users/0421362219be489e879b7c8988e4a2c9` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:28.712853 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0421362219be489e879b7c8988e4a2c9)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:28.721067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:28.779923 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0421362219be489e879b7c8988e4a2c9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:28.780656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0421362219be489e879b7c8988e4a2c9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:28.788194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0421362219be489e879b7c8988e4a2c9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:28.788932 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0421362219be489e879b7c8988e4a2c9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:28.800841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0421362219be489e879b7c8988e4a2c9'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:28.815248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1367/2718] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:28 2022] DELETE /identity/v3/users/0421362219be489e879b7c8988e4a2c9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:28.923423 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845595c6-998e-43e7-aa0a-db2ecec37d11 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:28.923710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845595c6-998e-43e7-aa0a-db2ecec37d11 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:28.923981 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845595c6-998e-43e7-aa0a-db2ecec37d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:28.943294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-845595c6-998e-43e7-aa0a-db2ecec37d11 None None] MFA Rules not processed for user `c710350e02634e84b3d4700ae5fa6925`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:28.944521 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-845595c6-998e-43e7-aa0a-db2ecec37d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:29.010817 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1352/2719] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:57:29.111811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:29.113354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:29.188553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1b6e5-ef6a-4414-a873-d7622adc9cc3 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:29.193776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:29.194400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:29.194681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:29.194953 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] PATH_INFO: `/v3/projects/d25b926ee1a943e08a1a9c36d835e6cd` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:29.196071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d25b926ee1a943e08a1a9c36d835e6cd)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:29.198060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:29.232937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:29.253422 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd25b926ee1a943e08a1a9c36d835e6cd', 'request_id': 'req-2a8dbbd0-8b71-478b-b458-85ac43ab870d'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:29.263537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd25b926ee1a943e08a1a9c36d835e6cd', 'request_id': 'req-2a8dbbd0-8b71-478b-b458-85ac43ab870d'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:29.270365 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd25b926ee1a943e08a1a9c36d835e6cd', 'request_id': 'req-2a8dbbd0-8b71-478b-b458-85ac43ab870d'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:29.282833 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] The token cache is being invalidate because project d25b926ee1a943e08a1a9c36d835e6cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:29.283130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:29.283784 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:29.285974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1368/2720] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:29 2022] DELETE /identity/v3/projects/d25b926ee1a943e08a1a9c36d835e6cd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:29.290110 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845595c6-998e-43e7-aa0a-db2ecec37d11 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:29.305769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845595c6-998e-43e7-aa0a-db2ecec37d11 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:29.376849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34baa784-fcb9-4891-aae8-8c1b383ed87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:29.377227 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34baa784-fcb9-4891-aae8-8c1b383ed87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:29.377517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34baa784-fcb9-4891-aae8-8c1b383ed87c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:29.377824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34baa784-fcb9-4891-aae8-8c1b383ed87c admin admin] PATH_INFO: `/v3/users/07c61abec08d4d258a0704cd15de8561` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:29.378965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34baa784-fcb9-4891-aae8-8c1b383ed87c admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c61abec08d4d258a0704cd15de8561)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:29.381201 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34baa784-fcb9-4891-aae8-8c1b383ed87c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:29.447533 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-34baa784-fcb9-4891-aae8-8c1b383ed87c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c61abec08d4d258a0704cd15de8561'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:29.448115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-34baa784-fcb9-4891-aae8-8c1b383ed87c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c61abec08d4d258a0704cd15de8561'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:29.462027 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:29.464816 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-34baa784-fcb9-4891-aae8-8c1b383ed87c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c61abec08d4d258a0704cd15de8561'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:29.480212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-34baa784-fcb9-4891-aae8-8c1b383ed87c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c61abec08d4d258a0704cd15de8561'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:29.481291 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-34baa784-fcb9-4891-aae8-8c1b383ed87c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c61abec08d4d258a0704cd15de8561'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:29.499259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1353/2721] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:29 2022] DELETE /identity/v3/users/07c61abec08d4d258a0704cd15de8561 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:29.828110 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:29.830264 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:29.854253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8dbbd0-8b71-478b-b458-85ac43ab870d admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:29.943373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:29.944137 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:29.944537 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:29.944700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52 admin admin] PATH_INFO: `/v3/projects/693962861175449387d03608387d5fd2` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:29.946047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=693962861175449387d03608387d5fd2)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:29.948366 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:30.006244 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '693962861175449387d03608387d5fd2', 'request_id': 'req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:30.013182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '693962861175449387d03608387d5fd2', 'request_id': 'req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:30.022728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '693962861175449387d03608387d5fd2', 'request_id': 'req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:30.036811 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52 admin admin] The token cache is being invalidate because project 693962861175449387d03608387d5fd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:30.037189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:30.037950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7ee34aad-3a9c-4fb9-94c2-bd5a2d5cbe52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:30.040295 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1369/2722] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:29 2022] DELETE /identity/v3/projects/693962861175449387d03608387d5fd2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:30.046074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab487b0-6ad3-411a-a759-d85f753d9ac1 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:30.046714 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab487b0-6ad3-411a-a759-d85f753d9ac1 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:30.047000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab487b0-6ad3-411a-a759-d85f753d9ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:30.072562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-6ab487b0-6ad3-411a-a759-d85f753d9ac1 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:30.074109 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6ab487b0-6ad3-411a-a759-d85f753d9ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:30.162049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1354/2723] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:30.170463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea4b99-26df-4f83-9b67-bb1a5fb1fc47 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:30.170759 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea4b99-26df-4f83-9b67-bb1a5fb1fc47 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:30.171035 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea4b99-26df-4f83-9b67-bb1a5fb1fc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:30.194041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-e9ea4b99-26df-4f83-9b67-bb1a5fb1fc47 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:30.195266 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e9ea4b99-26df-4f83-9b67-bb1a5fb1fc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:30.203347 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1370/2724] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:30.208475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab487b0-6ad3-411a-a759-d85f753d9ac1 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:30.219036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab487b0-6ad3-411a-a759-d85f753d9ac1 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:30.224490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9e304-383c-48fc-8054-7d03fae0f168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:30.225078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9e304-383c-48fc-8054-7d03fae0f168 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:30.225677 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9e304-383c-48fc-8054-7d03fae0f168 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:30.225957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9e304-383c-48fc-8054-7d03fae0f168 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:30.226683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9e304-383c-48fc-8054-7d03fae0f168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:57:30.226894 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9e304-383c-48fc-8054-7d03fae0f168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:30.228546 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a9e304-383c-48fc-8054-7d03fae0f168 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:30.245104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1355/2725] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:57:30.249056 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea4b99-26df-4f83-9b67-bb1a5fb1fc47 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:30.259628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea4b99-26df-4f83-9b67-bb1a5fb1fc47 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:30.264610 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a295cd0-9adc-482e-b0f0-82ea01959b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:30.265212 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a295cd0-9adc-482e-b0f0-82ea01959b99 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:30.265497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a295cd0-9adc-482e-b0f0-82ea01959b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:30.265802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a295cd0-9adc-482e-b0f0-82ea01959b99 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:30.266703 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a295cd0-9adc-482e-b0f0-82ea01959b99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:30.268560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a295cd0-9adc-482e-b0f0-82ea01959b99 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:30.306393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1371/2726] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:30 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:30.310680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9e304-383c-48fc-8054-7d03fae0f168 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:30.321657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9e304-383c-48fc-8054-7d03fae0f168 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:30.390261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378aaf79-9a00-4e28-a93c-dc78a8dc2598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:30.391380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378aaf79-9a00-4e28-a93c-dc78a8dc2598 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:30.391789 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378aaf79-9a00-4e28-a93c-dc78a8dc2598 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:30.392148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378aaf79-9a00-4e28-a93c-dc78a8dc2598 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:30.393107 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378aaf79-9a00-4e28-a93c-dc78a8dc2598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:30.396405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378aaf79-9a00-4e28-a93c-dc78a8dc2598 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:30.424290 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1356/2727] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:30 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 08:57:30.428664 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a295cd0-9adc-482e-b0f0-82ea01959b99 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:30.439805 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a295cd0-9adc-482e-b0f0-82ea01959b99 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:30.445553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7f153-1f44-4566-ae2d-164d0cc1d3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:30.446218 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7f153-1f44-4566-ae2d-164d0cc1d3ce admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:30.446406 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7f153-1f44-4566-ae2d-164d0cc1d3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:30.446675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7f153-1f44-4566-ae2d-164d0cc1d3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:30.447329 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7f153-1f44-4566-ae2d-164d0cc1d3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:30.448913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7f153-1f44-4566-ae2d-164d0cc1d3ce admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:30.469246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1372/2728] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:30.478193 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378aaf79-9a00-4e28-a93c-dc78a8dc2598 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:30.492783 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378aaf79-9a00-4e28-a93c-dc78a8dc2598 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:30.502076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bec0aa-b590-44d8-a69e-6e1598afcc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:30.502868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bec0aa-b590-44d8-a69e-6e1598afcc22 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:30.503387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bec0aa-b590-44d8-a69e-6e1598afcc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:30.504019 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bec0aa-b590-44d8-a69e-6e1598afcc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:30.505285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bec0aa-b590-44d8-a69e-6e1598afcc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:30.507222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bec0aa-b590-44d8-a69e-6e1598afcc22 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:30.521458 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1357/2729] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:30.525558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7f153-1f44-4566-ae2d-164d0cc1d3ce admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:30.532326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:30.537683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7f153-1f44-4566-ae2d-164d0cc1d3ce admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:30.547529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb09a94-68a9-439c-9c6a-d9f3eb096f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:30.548435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb09a94-68a9-439c-9c6a-d9f3eb096f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:30.548873 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb09a94-68a9-439c-9c6a-d9f3eb096f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:30.549429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb09a94-68a9-439c-9c6a-d9f3eb096f65 admin admin] PATH_INFO: `/v3/projects/ef387d705a9247a5b279fce570de03b7/users/5a23080d644d493a940849f4a323c6a5/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:30.567640 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb09a94-68a9-439c-9c6a-d9f3eb096f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef387d705a9247a5b279fce570de03b7, user_id=5a23080d644d493a940849f4a323c6a5, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:30.572078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb09a94-68a9-439c-9c6a-d9f3eb096f65 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:30.592681 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1373/2730] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:30 2022] PUT /identity/v3/projects/ef387d705a9247a5b279fce570de03b7/users/5a23080d644d493a940849f4a323c6a5/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:30.604774 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed760b-b3ec-4151-a650-c2a56c941153 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:30.605129 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed760b-b3ec-4151-a650-c2a56c941153 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:30.605471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed760b-b3ec-4151-a650-c2a56c941153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:30.640987 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-93ed760b-b3ec-4151-a650-c2a56c941153 None None] MFA Rules not processed for user `5a23080d644d493a940849f4a323c6a5`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:30.642195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-93ed760b-b3ec-4151-a650-c2a56c941153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:30.687077 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:30.720450 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1358/2731] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:30.733060 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb09a94-68a9-439c-9c6a-d9f3eb096f65 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:30.734919 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-acb09a94-68a9-439c-9c6a-d9f3eb096f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:30.797007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb09a94-68a9-439c-9c6a-d9f3eb096f65 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:30.801075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6061ca1-4c86-42d3-8f89-281c315b0b92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:30.801707 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6061ca1-4c86-42d3-8f89-281c315b0b92 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:30.802024 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6061ca1-4c86-42d3-8f89-281c315b0b92 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:30.802296 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6061ca1-4c86-42d3-8f89-281c315b0b92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:30.811938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6061ca1-4c86-42d3-8f89-281c315b0b92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:30.813386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6061ca1-4c86-42d3-8f89-281c315b0b92 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:30.823734 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1374/2732] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:40.361508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed760b-b3ec-4151-a650-c2a56c941153 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:40.423810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed760b-b3ec-4151-a650-c2a56c941153 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:40.428766 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:40.429387 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:40.429665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:40.429967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] PATH_INFO: `/v3/users/5a23080d644d493a940849f4a323c6a5` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:40.431073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a23080d644d493a940849f4a323c6a5)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:40.432974 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:40.477258 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a23080d644d493a940849f4a323c6a5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:40.477934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a23080d644d493a940849f4a323c6a5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:40.490564 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a23080d644d493a940849f4a323c6a5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:40.497178 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a23080d644d493a940849f4a323c6a5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:40.497841 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a23080d644d493a940849f4a323c6a5'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:40.509844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1359/2733] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:40 2022] DELETE /identity/v3/users/5a23080d644d493a940849f4a323c6a5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:40.514309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2d1fa-79af-46bd-b96a-be0699412863 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:40.514800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2d1fa-79af-46bd-b96a-be0699412863 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:40.514906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2d1fa-79af-46bd-b96a-be0699412863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:40.533100 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-c0a2d1fa-79af-46bd-b96a-be0699412863 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:40.534318 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c0a2d1fa-79af-46bd-b96a-be0699412863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:40.597167 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1375/2734] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:40.607647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:40.609086 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:40.666489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbe555-6636-448d-8177-8beff3791c1b admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:40.670251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311015b1-0f40-4f3c-a240-f74787c2dc92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:40.670800 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311015b1-0f40-4f3c-a240-f74787c2dc92 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:40.671088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311015b1-0f40-4f3c-a240-f74787c2dc92 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:40.671364 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311015b1-0f40-4f3c-a240-f74787c2dc92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:40.678849 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:40.681474 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311015b1-0f40-4f3c-a240-f74787c2dc92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:40.683871 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311015b1-0f40-4f3c-a240-f74787c2dc92 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:40.694162 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1360/2735] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 08:57:40.955414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2d1fa-79af-46bd-b96a-be0699412863 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:40.956824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c0a2d1fa-79af-46bd-b96a-be0699412863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:40.969165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2d1fa-79af-46bd-b96a-be0699412863 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:40.973843 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44a893-2ad2-4680-a3ef-472a5f5cf557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:40.975079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44a893-2ad2-4680-a3ef-472a5f5cf557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:40.975352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44a893-2ad2-4680-a3ef-472a5f5cf557 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:40.975621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44a893-2ad2-4680-a3ef-472a5f5cf557 admin admin] PATH_INFO: `/v3/projects/ef387d705a9247a5b279fce570de03b7` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:40.976854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44a893-2ad2-4680-a3ef-472a5f5cf557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef387d705a9247a5b279fce570de03b7)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:40.978676 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44a893-2ad2-4680-a3ef-472a5f5cf557 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:41.028375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6f44a893-2ad2-4680-a3ef-472a5f5cf557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef387d705a9247a5b279fce570de03b7', 'request_id': 'req-6f44a893-2ad2-4680-a3ef-472a5f5cf557'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:41.035504 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6f44a893-2ad2-4680-a3ef-472a5f5cf557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef387d705a9247a5b279fce570de03b7', 'request_id': 'req-6f44a893-2ad2-4680-a3ef-472a5f5cf557'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:41.041613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6f44a893-2ad2-4680-a3ef-472a5f5cf557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef387d705a9247a5b279fce570de03b7', 'request_id': 'req-6f44a893-2ad2-4680-a3ef-472a5f5cf557'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:41.053817 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6f44a893-2ad2-4680-a3ef-472a5f5cf557 admin admin] The token cache is being invalidate because project ef387d705a9247a5b279fce570de03b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:41.054109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6f44a893-2ad2-4680-a3ef-472a5f5cf557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:41.054730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6f44a893-2ad2-4680-a3ef-472a5f5cf557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:41.056227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1376/2736] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:40 2022] DELETE /identity/v3/projects/ef387d705a9247a5b279fce570de03b7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:41.061999 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160403e6-9771-465b-9c06-dda339f8cc4e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.062285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160403e6-9771-465b-9c06-dda339f8cc4e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.062607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160403e6-9771-465b-9c06-dda339f8cc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.082032 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-160403e6-9771-465b-9c06-dda339f8cc4e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:41.083971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-160403e6-9771-465b-9c06-dda339f8cc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:41.141524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1361/2737] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:41.204515 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7ec9e-5af0-49f9-af04-7781b800b37b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.204733 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7ec9e-5af0-49f9-af04-7781b800b37b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.204971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7ec9e-5af0-49f9-af04-7781b800b37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.220422 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:41.226210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-f2d7ec9e-5af0-49f9-af04-7781b800b37b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:41.227979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f2d7ec9e-5af0-49f9-af04-7781b800b37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:41.237399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1377/2738] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:41.241681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160403e6-9771-465b-9c06-dda339f8cc4e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:41.251180 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160403e6-9771-465b-9c06-dda339f8cc4e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:41.255689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9455f-fa64-41f0-8f96-722a15dd1df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:41.256274 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9455f-fa64-41f0-8f96-722a15dd1df4 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.256549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9455f-fa64-41f0-8f96-722a15dd1df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.257345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c9455f-fa64-41f0-8f96-722a15dd1df4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.258009 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9455f-fa64-41f0-8f96-722a15dd1df4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:57:41.258320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9455f-fa64-41f0-8f96-722a15dd1df4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:41.259886 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c9455f-fa64-41f0-8f96-722a15dd1df4 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:41.271060 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1362/2739] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:57:41.275872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7ec9e-5af0-49f9-af04-7781b800b37b None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:41.284918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7ec9e-5af0-49f9-af04-7781b800b37b None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:41.289670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeecf55-2db9-4406-883c-80d881dd0f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:41.290293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeecf55-2db9-4406-883c-80d881dd0f50 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.290575 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeecf55-2db9-4406-883c-80d881dd0f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.290847 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeecf55-2db9-4406-883c-80d881dd0f50 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.291467 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeecf55-2db9-4406-883c-80d881dd0f50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:41.293364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeecf55-2db9-4406-883c-80d881dd0f50 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:41.327984 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1378/2740] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:41 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:41.330261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9455f-fa64-41f0-8f96-722a15dd1df4 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:41.341456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c9455f-fa64-41f0-8f96-722a15dd1df4 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:41.414207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879a815-3515-4178-ae63-9c5c473a0ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:41.414954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879a815-3515-4178-ae63-9c5c473a0ffe admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.415207 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879a815-3515-4178-ae63-9c5c473a0ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.415364 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879a815-3515-4178-ae63-9c5c473a0ffe admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.416038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879a815-3515-4178-ae63-9c5c473a0ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:41.418015 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879a815-3515-4178-ae63-9c5c473a0ffe admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:41.449743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1363/2741] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:41 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:41.450998 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeecf55-2db9-4406-883c-80d881dd0f50 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:41.463599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeecf55-2db9-4406-883c-80d881dd0f50 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:41.469125 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a39e4-6e5c-491b-a276-948538e60266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:41.469955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a39e4-6e5c-491b-a276-948538e60266 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.470241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a39e4-6e5c-491b-a276-948538e60266 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.470513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a39e4-6e5c-491b-a276-948538e60266 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.471165 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a39e4-6e5c-491b-a276-948538e60266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:41.472791 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a39e4-6e5c-491b-a276-948538e60266 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:41.495951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1379/2742] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:41.499893 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879a815-3515-4178-ae63-9c5c473a0ffe admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:41.517667 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879a815-3515-4178-ae63-9c5c473a0ffe admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:41.523158 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006c4e7-7858-446e-a089-15a6dc5cf476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:41.523759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006c4e7-7858-446e-a089-15a6dc5cf476 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.524036 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006c4e7-7858-446e-a089-15a6dc5cf476 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.524306 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006c4e7-7858-446e-a089-15a6dc5cf476 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.524947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006c4e7-7858-446e-a089-15a6dc5cf476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:41.526555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006c4e7-7858-446e-a089-15a6dc5cf476 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:41.536948 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1364/2743] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:41.541423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a39e4-6e5c-491b-a276-948538e60266 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:41.551771 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a39e4-6e5c-491b-a276-948538e60266 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:41.557635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376a1cd-8188-4e42-9d98-139548e04eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:41.558351 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376a1cd-8188-4e42-9d98-139548e04eea admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.558628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376a1cd-8188-4e42-9d98-139548e04eea admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.558902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376a1cd-8188-4e42-9d98-139548e04eea admin admin] PATH_INFO: `/v3/projects/62e6f477ddbe42f5a5c82a44b420e504/users/8f83c84f86994658874086a36aca36cb/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.574855 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376a1cd-8188-4e42-9d98-139548e04eea admin admin] RBAC: Authorizing `identity:create_grant(project_id=62e6f477ddbe42f5a5c82a44b420e504, user_id=8f83c84f86994658874086a36aca36cb, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:41.577517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4376a1cd-8188-4e42-9d98-139548e04eea admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:41.591306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1380/2744] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:41 2022] PUT /identity/v3/projects/62e6f477ddbe42f5a5c82a44b420e504/users/8f83c84f86994658874086a36aca36cb/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:41.601361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1311a86-1228-4b44-a0da-fdd342e0030f None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.601779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1311a86-1228-4b44-a0da-fdd342e0030f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.602195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1311a86-1228-4b44-a0da-fdd342e0030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.631617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-a1311a86-1228-4b44-a0da-fdd342e0030f None None] MFA Rules not processed for user `8f83c84f86994658874086a36aca36cb`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:41.633556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a1311a86-1228-4b44-a0da-fdd342e0030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:41.687410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:41.706099 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1365/2745] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:41.710922 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376a1cd-8188-4e42-9d98-139548e04eea admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:41.774408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4376a1cd-8188-4e42-9d98-139548e04eea admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:41.779046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092e392-7bf7-4618-9d69-af32c84b80c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:41.779643 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092e392-7bf7-4618-9d69-af32c84b80c7 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.779920 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092e392-7bf7-4618-9d69-af32c84b80c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.780189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092e392-7bf7-4618-9d69-af32c84b80c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.780820 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092e392-7bf7-4618-9d69-af32c84b80c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:41.782744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092e392-7bf7-4618-9d69-af32c84b80c7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:41.808893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1381/2746] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:41 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:41.813063 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1311a86-1228-4b44-a0da-fdd342e0030f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:41.824123 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1311a86-1228-4b44-a0da-fdd342e0030f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:41.878487 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c6c03-a4d7-4fc6-b43c-cf6b978b4cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:41.879154 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c6c03-a4d7-4fc6-b43c-cf6b978b4cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.879435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c6c03-a4d7-4fc6-b43c-cf6b978b4cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.879707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c6c03-a4d7-4fc6-b43c-cf6b978b4cc4 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.880324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c6c03-a4d7-4fc6-b43c-cf6b978b4cc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:41.882943 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c6c03-a4d7-4fc6-b43c-cf6b978b4cc4 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:41.905516 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1366/2747] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:41 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:41.906725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092e392-7bf7-4618-9d69-af32c84b80c7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:41.917858 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092e392-7bf7-4618-9d69-af32c84b80c7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:41.922460 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fb7ad-5be7-45aa-b91d-dc450738f27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:41.923064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fb7ad-5be7-45aa-b91d-dc450738f27a admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.923342 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fb7ad-5be7-45aa-b91d-dc450738f27a admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.923617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300fb7ad-5be7-45aa-b91d-dc450738f27a admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.924258 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fb7ad-5be7-45aa-b91d-dc450738f27a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:41.925888 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300fb7ad-5be7-45aa-b91d-dc450738f27a admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:41.936928 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1382/2748] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:41.940812 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c6c03-a4d7-4fc6-b43c-cf6b978b4cc4 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:41.950662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c6c03-a4d7-4fc6-b43c-cf6b978b4cc4 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:41.955345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8028b97-d8cb-43ed-b31d-0bd13fa75afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:41.955960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8028b97-d8cb-43ed-b31d-0bd13fa75afc admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.956237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8028b97-d8cb-43ed-b31d-0bd13fa75afc admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.956507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8028b97-d8cb-43ed-b31d-0bd13fa75afc admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:41.957140 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8028b97-d8cb-43ed-b31d-0bd13fa75afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:41.958994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8028b97-d8cb-43ed-b31d-0bd13fa75afc admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:41.968620 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1367/2749] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:41.972675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fb7ad-5be7-45aa-b91d-dc450738f27a admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:41.983166 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300fb7ad-5be7-45aa-b91d-dc450738f27a admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:41.987774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b1fed-4de8-40d9-bece-0005a991fb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:41.988417 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b1fed-4de8-40d9-bece-0005a991fb8e admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:41.988696 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b1fed-4de8-40d9-bece-0005a991fb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:41.988970 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b1fed-4de8-40d9-bece-0005a991fb8e admin admin] PATH_INFO: `/v3/projects/e76cfd85e65546f197f1e8218be5ccd2/users/14a2731565e54360a6cbd339b82045da/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:42.005006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b1fed-4de8-40d9-bece-0005a991fb8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e76cfd85e65546f197f1e8218be5ccd2, user_id=14a2731565e54360a6cbd339b82045da, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:42.008413 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b1fed-4de8-40d9-bece-0005a991fb8e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:42.020694 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1383/2750] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:41 2022] PUT /identity/v3/projects/e76cfd85e65546f197f1e8218be5ccd2/users/14a2731565e54360a6cbd339b82045da/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:42.031540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca2b0b-0134-4fa2-9b2d-4aef407e6575 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:42.031857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca2b0b-0134-4fa2-9b2d-4aef407e6575 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:42.032220 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca2b0b-0134-4fa2-9b2d-4aef407e6575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:42.060384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-40ca2b0b-0134-4fa2-9b2d-4aef407e6575 None None] MFA Rules not processed for user `14a2731565e54360a6cbd339b82045da`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:42.061585 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-40ca2b0b-0134-4fa2-9b2d-4aef407e6575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:42.126457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1368/2751] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:57:42.130824 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b1fed-4de8-40d9-bece-0005a991fb8e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:42.183636 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:42.195887 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b1fed-4de8-40d9-bece-0005a991fb8e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:42.201178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbc752-7e12-47a0-b6bf-1549204646ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:42.201813 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbc752-7e12-47a0-b6bf-1549204646ee admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:42.202161 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbc752-7e12-47a0-b6bf-1549204646ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:42.202449 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbc752-7e12-47a0-b6bf-1549204646ee admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:42.203002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbc752-7e12-47a0-b6bf-1549204646ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:42.204910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbc752-7e12-47a0-b6bf-1549204646ee admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:42.232471 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1384/2752] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:42 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:42.236505 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca2b0b-0134-4fa2-9b2d-4aef407e6575 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:42.246960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca2b0b-0134-4fa2-9b2d-4aef407e6575 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:42.301784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f60a6d-89c8-4c1f-a024-8759b4b9e2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:42.302322 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f60a6d-89c8-4c1f-a024-8759b4b9e2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:42.302599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f60a6d-89c8-4c1f-a024-8759b4b9e2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:42.302867 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f60a6d-89c8-4c1f-a024-8759b4b9e2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:42.303490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f60a6d-89c8-4c1f-a024-8759b4b9e2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:42.305934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f60a6d-89c8-4c1f-a024-8759b4b9e2f1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:42.313607 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:42.327372 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1369/2753] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:42 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:42.331395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbc752-7e12-47a0-b6bf-1549204646ee admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:42.342138 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbc752-7e12-47a0-b6bf-1549204646ee admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:42.346711 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc025585-cfbf-4610-98ba-102011820996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:42.347288 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025585-cfbf-4610-98ba-102011820996 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:42.347565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025585-cfbf-4610-98ba-102011820996 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:42.347837 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025585-cfbf-4610-98ba-102011820996 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:42.348470 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc025585-cfbf-4610-98ba-102011820996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:42.350116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc025585-cfbf-4610-98ba-102011820996 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:42.360721 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1385/2754] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:42.364611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f60a6d-89c8-4c1f-a024-8759b4b9e2f1 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:42.375259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f60a6d-89c8-4c1f-a024-8759b4b9e2f1 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:42.379887 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a356135-60fd-4ec4-b536-a6e288932ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:42.380577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a356135-60fd-4ec4-b536-a6e288932ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:42.380857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a356135-60fd-4ec4-b536-a6e288932ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:42.381131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a356135-60fd-4ec4-b536-a6e288932ad1 admin admin] PATH_INFO: `/v3/projects/63415626c5f44380aa7cee036e93628f/users/e6f39000d4164bc981c05b3639cc3810/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:42.397511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a356135-60fd-4ec4-b536-a6e288932ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63415626c5f44380aa7cee036e93628f, user_id=e6f39000d4164bc981c05b3639cc3810, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:42.400175 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a356135-60fd-4ec4-b536-a6e288932ad1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:42.413504 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1370/2755] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:42 2022] PUT /identity/v3/projects/63415626c5f44380aa7cee036e93628f/users/e6f39000d4164bc981c05b3639cc3810/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:42.417407 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc025585-cfbf-4610-98ba-102011820996 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:42.427374 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc025585-cfbf-4610-98ba-102011820996 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:42.482527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b31535-0156-49a8-960b-6a6072a1b7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:42.483125 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b31535-0156-49a8-960b-6a6072a1b7ff admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:42.483408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b31535-0156-49a8-960b-6a6072a1b7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:42.483685 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b31535-0156-49a8-960b-6a6072a1b7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:42.484487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b31535-0156-49a8-960b-6a6072a1b7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:42.486203 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b31535-0156-49a8-960b-6a6072a1b7ff admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:42.496373 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1386/2756] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:42.500514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a356135-60fd-4ec4-b536-a6e288932ad1 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:42.511241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a356135-60fd-4ec4-b536-a6e288932ad1 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:42.516034 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c003c-04c5-40c9-9e25-770037b15cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:42.516810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c003c-04c5-40c9-9e25-770037b15cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:42.516945 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c003c-04c5-40c9-9e25-770037b15cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:42.517294 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c003c-04c5-40c9-9e25-770037b15cc4 admin admin] PATH_INFO: `/v3/projects/63415626c5f44380aa7cee036e93628f/users/e6f39000d4164bc981c05b3639cc3810/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:42.519151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c003c-04c5-40c9-9e25-770037b15cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63415626c5f44380aa7cee036e93628f, user_id=e6f39000d4164bc981c05b3639cc3810, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:42.521788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c003c-04c5-40c9-9e25-770037b15cc4 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:42.532074 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1371/2757] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:42 2022] PUT /identity/v3/projects/63415626c5f44380aa7cee036e93628f/users/e6f39000d4164bc981c05b3639cc3810/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:42.543630 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7bf21-0928-41b9-87ba-61cf3b62bf2d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:42.544162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7bf21-0928-41b9-87ba-61cf3b62bf2d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:42.544162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7bf21-0928-41b9-87ba-61cf3b62bf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:42.573122 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-06c7bf21-0928-41b9-87ba-61cf3b62bf2d None None] MFA Rules not processed for user `e6f39000d4164bc981c05b3639cc3810`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:42.574287 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-06c7bf21-0928-41b9-87ba-61cf3b62bf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:42.645738 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1387/2758] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:42.657373 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c003c-04c5-40c9-9e25-770037b15cc4 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:42.659780 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6f9c003c-04c5-40c9-9e25-770037b15cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:42.717308 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c003c-04c5-40c9-9e25-770037b15cc4 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:42.720881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4131444-b17d-4f8e-a105-cf47ade97217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:42.721472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4131444-b17d-4f8e-a105-cf47ade97217 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:42.721773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4131444-b17d-4f8e-a105-cf47ade97217 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:42.722044 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4131444-b17d-4f8e-a105-cf47ade97217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:42.776016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4131444-b17d-4f8e-a105-cf47ade97217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:42.777436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4131444-b17d-4f8e-a105-cf47ade97217 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:42.786070 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1372/2759] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:44.277743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7bf21-0928-41b9-87ba-61cf3b62bf2d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:44.279234 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-06c7bf21-0928-41b9-87ba-61cf3b62bf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:44.291560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7bf21-0928-41b9-87ba-61cf3b62bf2d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:44.295033 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4080107-9607-439a-a267-61d33a52115a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:44.295621 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4080107-9607-439a-a267-61d33a52115a service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:44.295926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4080107-9607-439a-a267-61d33a52115a service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:44.296197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4080107-9607-439a-a267-61d33a52115a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:44.339507 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:44.352249 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4080107-9607-439a-a267-61d33a52115a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:44.353618 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4080107-9607-439a-a267-61d33a52115a service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:44.362271 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1388/2760] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:44.866279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4131444-b17d-4f8e-a105-cf47ade97217 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:44.875198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4131444-b17d-4f8e-a105-cf47ade97217 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:44.878765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642f9ac-972f-4ac1-a93c-2ff4bbf12762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:44.879362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642f9ac-972f-4ac1-a93c-2ff4bbf12762 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:44.879637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642f9ac-972f-4ac1-a93c-2ff4bbf12762 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:44.879934 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642f9ac-972f-4ac1-a93c-2ff4bbf12762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:44.889517 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642f9ac-972f-4ac1-a93c-2ff4bbf12762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:44.891067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642f9ac-972f-4ac1-a93c-2ff4bbf12762 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:44.901531 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1373/2761] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:48.823148 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4080107-9607-439a-a267-61d33a52115a service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:48.883956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4080107-9607-439a-a267-61d33a52115a service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:48.888448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:48.889067 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:48.889346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:48.889618 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] PATH_INFO: `/v3/users/8f83c84f86994658874086a36aca36cb` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:48.890764 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f83c84f86994658874086a36aca36cb)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:48.892739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:48.936969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f83c84f86994658874086a36aca36cb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:48.937614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f83c84f86994658874086a36aca36cb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:48.944903 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f83c84f86994658874086a36aca36cb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:48.945533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f83c84f86994658874086a36aca36cb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:48.957235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f83c84f86994658874086a36aca36cb'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:48.970290 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1389/2762] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:48 2022] DELETE /identity/v3/users/8f83c84f86994658874086a36aca36cb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:48.974721 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d58e2b-fe4b-4ccc-906d-2a6b062de855 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:48.975179 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d58e2b-fe4b-4ccc-906d-2a6b062de855 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:48.975297 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d58e2b-fe4b-4ccc-906d-2a6b062de855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:48.993527 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-65d58e2b-fe4b-4ccc-906d-2a6b062de855 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:48.994745 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-65d58e2b-fe4b-4ccc-906d-2a6b062de855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:49.040368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:49.053486 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1374/2763] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:49.063499 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:49.064910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:49.120772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0700d5c-affd-4a59-83e7-9cc4015bdbb4 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:49.124378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257475b-ac1e-40f2-a59b-2934b54e65fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:49.124963 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257475b-ac1e-40f2-a59b-2934b54e65fd service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:49.125241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257475b-ac1e-40f2-a59b-2934b54e65fd service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:49.125508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257475b-ac1e-40f2-a59b-2934b54e65fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:49.134885 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257475b-ac1e-40f2-a59b-2934b54e65fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:49.136304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a257475b-ac1e-40f2-a59b-2934b54e65fd service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:49.145526 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1390/2764] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:57:49.408967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d58e2b-fe4b-4ccc-906d-2a6b062de855 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:49.410441 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-65d58e2b-fe4b-4ccc-906d-2a6b062de855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:49.424735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d58e2b-fe4b-4ccc-906d-2a6b062de855 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:49.439416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:49.440104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:49.440278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:49.440548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] PATH_INFO: `/v3/projects/62e6f477ddbe42f5a5c82a44b420e504` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:49.443749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=62e6f477ddbe42f5a5c82a44b420e504)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:49.443749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:49.492374 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62e6f477ddbe42f5a5c82a44b420e504', 'request_id': 'req-b7f983d6-4215-4563-b246-6fbb11dc76fa'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:49.498500 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62e6f477ddbe42f5a5c82a44b420e504', 'request_id': 'req-b7f983d6-4215-4563-b246-6fbb11dc76fa'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:49.504081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62e6f477ddbe42f5a5c82a44b420e504', 'request_id': 'req-b7f983d6-4215-4563-b246-6fbb11dc76fa'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:49.516283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] The token cache is being invalidate because project 62e6f477ddbe42f5a5c82a44b420e504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:49.516575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:49.517208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:49.518698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1375/2765] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:49 2022] DELETE /identity/v3/projects/62e6f477ddbe42f5a5c82a44b420e504 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:57:49.522850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257475b-ac1e-40f2-a59b-2934b54e65fd service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:49.532253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257475b-ac1e-40f2-a59b-2934b54e65fd service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:49.536959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:49.590273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:49.590937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:49.591207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:49.591482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] PATH_INFO: `/v3/users/14a2731565e54360a6cbd339b82045da` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:49.592594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14a2731565e54360a6cbd339b82045da)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:49.594558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:49.646054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14a2731565e54360a6cbd339b82045da'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:49.646716 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14a2731565e54360a6cbd339b82045da'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:49.653908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14a2731565e54360a6cbd339b82045da'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:49.654616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14a2731565e54360a6cbd339b82045da'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:49.667190 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14a2731565e54360a6cbd339b82045da'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:49.680106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1391/2766] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:49 2022] DELETE /identity/v3/users/14a2731565e54360a6cbd339b82045da => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:49.944729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:49.946404 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:49.963041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f983d6-4215-4563-b246-6fbb11dc76fa admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:50.025416 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:50.026085 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:50.026335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:50.026602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] PATH_INFO: `/v3/projects/e76cfd85e65546f197f1e8218be5ccd2` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:50.027737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e76cfd85e65546f197f1e8218be5ccd2)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:50.029629 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:50.073958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e76cfd85e65546f197f1e8218be5ccd2', 'request_id': 'req-3923cffa-c0cd-4a6c-84f7-3656391084fd'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.079828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e76cfd85e65546f197f1e8218be5ccd2', 'request_id': 'req-3923cffa-c0cd-4a6c-84f7-3656391084fd'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.086062 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e76cfd85e65546f197f1e8218be5ccd2', 'request_id': 'req-3923cffa-c0cd-4a6c-84f7-3656391084fd'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.098010 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] The token cache is being invalidate because project e76cfd85e65546f197f1e8218be5ccd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:50.098295 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.098907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.100348 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1376/2767] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:49 2022] DELETE /identity/v3/projects/e76cfd85e65546f197f1e8218be5ccd2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:50.105045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:50.106469 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:50.119077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202a4eb-63cf-4821-827b-d4b2920a7901 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:50.176181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f973f-26f1-4272-9f8b-f5298554d486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:50.176800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f973f-26f1-4272-9f8b-f5298554d486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:50.177080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f973f-26f1-4272-9f8b-f5298554d486 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:50.177353 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f973f-26f1-4272-9f8b-f5298554d486 admin admin] PATH_INFO: `/v3/users/e6f39000d4164bc981c05b3639cc3810` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:50.178490 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f973f-26f1-4272-9f8b-f5298554d486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f39000d4164bc981c05b3639cc3810)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:50.180455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f973f-26f1-4272-9f8b-f5298554d486 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:50.214587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:50.228883 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a25f973f-26f1-4272-9f8b-f5298554d486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f39000d4164bc981c05b3639cc3810'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.229527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a25f973f-26f1-4272-9f8b-f5298554d486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f39000d4164bc981c05b3639cc3810'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.236291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a25f973f-26f1-4272-9f8b-f5298554d486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f39000d4164bc981c05b3639cc3810'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.236913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a25f973f-26f1-4272-9f8b-f5298554d486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f39000d4164bc981c05b3639cc3810'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.248431 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a25f973f-26f1-4272-9f8b-f5298554d486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f39000d4164bc981c05b3639cc3810'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.261292 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1392/2768] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:57:50 2022] DELETE /identity/v3/users/e6f39000d4164bc981c05b3639cc3810 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 08:57:50.529073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:50.530609 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:50.543712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923cffa-c0cd-4a6c-84f7-3656391084fd admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:50.547639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:50.603434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bde6f-8074-46be-8980-078fd6a3b28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:50.604064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bde6f-8074-46be-8980-078fd6a3b28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:50.604343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bde6f-8074-46be-8980-078fd6a3b28e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:50.604615 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bde6f-8074-46be-8980-078fd6a3b28e admin admin] PATH_INFO: `/v3/projects/63415626c5f44380aa7cee036e93628f` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:50.605738 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bde6f-8074-46be-8980-078fd6a3b28e admin admin] RBAC: Authorizing `identity:delete_project(project_id=63415626c5f44380aa7cee036e93628f)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:50.607685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bde6f-8074-46be-8980-078fd6a3b28e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:50.651422 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b89bde6f-8074-46be-8980-078fd6a3b28e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63415626c5f44380aa7cee036e93628f', 'request_id': 'req-b89bde6f-8074-46be-8980-078fd6a3b28e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.657540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b89bde6f-8074-46be-8980-078fd6a3b28e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63415626c5f44380aa7cee036e93628f', 'request_id': 'req-b89bde6f-8074-46be-8980-078fd6a3b28e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.664115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b89bde6f-8074-46be-8980-078fd6a3b28e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63415626c5f44380aa7cee036e93628f', 'request_id': 'req-b89bde6f-8074-46be-8980-078fd6a3b28e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.676649 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b89bde6f-8074-46be-8980-078fd6a3b28e admin admin] The token cache is being invalidate because project 63415626c5f44380aa7cee036e93628f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:57:50.676935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b89bde6f-8074-46be-8980-078fd6a3b28e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.677545 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b89bde6f-8074-46be-8980-078fd6a3b28e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:57:50.679064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1377/2769] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:57:50 2022] DELETE /identity/v3/projects/63415626c5f44380aa7cee036e93628f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:50.684863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f05fc9-c6b5-44e5-9d84-08edecc2cc54 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:50.685149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f05fc9-c6b5-44e5-9d84-08edecc2cc54 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:50.685419 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f05fc9-c6b5-44e5-9d84-08edecc2cc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:50.704718 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-50f05fc9-c6b5-44e5-9d84-08edecc2cc54 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:50.706002 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-50f05fc9-c6b5-44e5-9d84-08edecc2cc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:50.763634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1393/2770] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:57:50.771440 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adcc74-f9ef-4178-8e42-b00734db9c14 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:50.771729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adcc74-f9ef-4178-8e42-b00734db9c14 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:50.772007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adcc74-f9ef-4178-8e42-b00734db9c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:50.792225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f3adcc74-f9ef-4178-8e42-b00734db9c14 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:50.793418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f3adcc74-f9ef-4178-8e42-b00734db9c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:50.801175 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1378/2771] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:50.805341 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f05fc9-c6b5-44e5-9d84-08edecc2cc54 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:50.816416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f05fc9-c6b5-44e5-9d84-08edecc2cc54 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:50.822687 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b4311-38b6-4434-b2e5-0a5e73e1fac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:50.823652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b4311-38b6-4434-b2e5-0a5e73e1fac7 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:50.824336 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b4311-38b6-4434-b2e5-0a5e73e1fac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:50.824611 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b4311-38b6-4434-b2e5-0a5e73e1fac7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:50.825254 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b4311-38b6-4434-b2e5-0a5e73e1fac7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 08:57:50.825556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b4311-38b6-4434-b2e5-0a5e73e1fac7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:50.827199 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b4311-38b6-4434-b2e5-0a5e73e1fac7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:50.839295 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1394/2772] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 08:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 08:57:50.843748 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adcc74-f9ef-4178-8e42-b00734db9c14 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:50.853351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adcc74-f9ef-4178-8e42-b00734db9c14 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:50.857962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019f610-031e-42ed-b862-b89d5b7b4132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:50.858540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019f610-031e-42ed-b862-b89d5b7b4132 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:50.858814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019f610-031e-42ed-b862-b89d5b7b4132 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:50.859079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019f610-031e-42ed-b862-b89d5b7b4132 admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:50.859710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019f610-031e-42ed-b862-b89d5b7b4132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:50.861580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019f610-031e-42ed-b862-b89d5b7b4132 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:50.887685 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1379/2773] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 08:57:50 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:50.891729 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b4311-38b6-4434-b2e5-0a5e73e1fac7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:50.902488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b4311-38b6-4434-b2e5-0a5e73e1fac7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:50.959830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791562a-eebd-48ac-9765-5e85aafc2522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:50.960424 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791562a-eebd-48ac-9765-5e85aafc2522 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:50.960706 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791562a-eebd-48ac-9765-5e85aafc2522 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:50.960980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791562a-eebd-48ac-9765-5e85aafc2522 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:50.961607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791562a-eebd-48ac-9765-5e85aafc2522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:50.963548 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791562a-eebd-48ac-9765-5e85aafc2522 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:50.984913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1395/2774] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 08:57:50 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 08:57:50.989028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019f610-031e-42ed-b862-b89d5b7b4132 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:50.999603 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019f610-031e-42ed-b862-b89d5b7b4132 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:51.004093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66830681-8022-4d3f-987c-e653097b60e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:51.004690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66830681-8022-4d3f-987c-e653097b60e3 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:51.004967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66830681-8022-4d3f-987c-e653097b60e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:51.005239 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66830681-8022-4d3f-987c-e653097b60e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:51.005912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66830681-8022-4d3f-987c-e653097b60e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:51.007486 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66830681-8022-4d3f-987c-e653097b60e3 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:51.017022 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1380/2775] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:51.020776 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791562a-eebd-48ac-9765-5e85aafc2522 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:51.030651 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791562a-eebd-48ac-9765-5e85aafc2522 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:51.035247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbdd72c-56c3-4f85-9cc6-d1a5ef093a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:51.035835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdd72c-56c3-4f85-9cc6-d1a5ef093a30 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:51.036116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdd72c-56c3-4f85-9cc6-d1a5ef093a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:51.036389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdd72c-56c3-4f85-9cc6-d1a5ef093a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:51.037023 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbdd72c-56c3-4f85-9cc6-d1a5ef093a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:51.039266 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbdd72c-56c3-4f85-9cc6-d1a5ef093a30 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:51.048974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1396/2776] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 08:57:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 08:57:51.052707 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66830681-8022-4d3f-987c-e653097b60e3 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:51.062901 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66830681-8022-4d3f-987c-e653097b60e3 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:51.067435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41cb72-7a1e-42be-878b-a547d0b9d60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:51.068055 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41cb72-7a1e-42be-878b-a547d0b9d60d admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:51.068331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41cb72-7a1e-42be-878b-a547d0b9d60d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:51.068600 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41cb72-7a1e-42be-878b-a547d0b9d60d admin admin] PATH_INFO: `/v3/projects/d2ebd9326b1e41b7a4a5a86f8b770c0b/users/e15e8b4bea2b4426b9384b0bc57e1ca1/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:51.082508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41cb72-7a1e-42be-878b-a547d0b9d60d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ebd9326b1e41b7a4a5a86f8b770c0b, user_id=e15e8b4bea2b4426b9384b0bc57e1ca1, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:51.085380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41cb72-7a1e-42be-878b-a547d0b9d60d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:51.098283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1381/2777] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 08:57:51 2022] PUT /identity/v3/projects/d2ebd9326b1e41b7a4a5a86f8b770c0b/users/e15e8b4bea2b4426b9384b0bc57e1ca1/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:57:51.110520 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b6541-e6a0-4265-8eb7-c72b6188919b None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:51.111118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b6541-e6a0-4265-8eb7-c72b6188919b None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:51.111118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b6541-e6a0-4265-8eb7-c72b6188919b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:51.136763 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:51.140488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-731b6541-e6a0-4265-8eb7-c72b6188919b None None] MFA Rules not processed for user `e15e8b4bea2b4426b9384b0bc57e1ca1`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:57:51.142607 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-731b6541-e6a0-4265-8eb7-c72b6188919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:51.206609 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1397/2778] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:57:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 08:57:51.217412 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41cb72-7a1e-42be-878b-a547d0b9d60d admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:57:51.219253 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2a41cb72-7a1e-42be-878b-a547d0b9d60d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:57:51.253635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:57:51.276985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41cb72-7a1e-42be-878b-a547d0b9d60d admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:57:51.280567 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb345f9-9901-4682-8613-13abcac571e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:57:51.281165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb345f9-9901-4682-8613-13abcac571e5 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:57:51.281444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb345f9-9901-4682-8613-13abcac571e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:57:51.281738 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb345f9-9901-4682-8613-13abcac571e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:57:51.290353 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb345f9-9901-4682-8613-13abcac571e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:57:51.291769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb345f9-9901-4682-8613-13abcac571e5 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:57:51.300231 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1382/2779] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:57:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:58:01.150437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b6541-e6a0-4265-8eb7-c72b6188919b None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:01.211149 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b6541-e6a0-4265-8eb7-c72b6188919b None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:01.216139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:01.216766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:01.217049 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:01.217326 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] PATH_INFO: `/v3/users/e15e8b4bea2b4426b9384b0bc57e1ca1` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:01.218476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e15e8b4bea2b4426b9384b0bc57e1ca1)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:01.220402 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:01.264321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e15e8b4bea2b4426b9384b0bc57e1ca1'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:01.264976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e15e8b4bea2b4426b9384b0bc57e1ca1'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:01.271770 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e15e8b4bea2b4426b9384b0bc57e1ca1'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:01.272409 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e15e8b4bea2b4426b9384b0bc57e1ca1'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:01.284571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e15e8b4bea2b4426b9384b0bc57e1ca1'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:01.296341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=11 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:01.296809 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1398/2780] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:58:01 2022] DELETE /identity/v3/users/e15e8b4bea2b4426b9384b0bc57e1ca1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:58:01.301245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a712804-b7c5-477d-818e-7147edd8894a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:01.301578 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a712804-b7c5-477d-818e-7147edd8894a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:01.301881 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a712804-b7c5-477d-818e-7147edd8894a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:01.320342 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-0a712804-b7c5-477d-818e-7147edd8894a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 08:58:01.321514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0a712804-b7c5-477d-818e-7147edd8894a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:01.378610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1383/2781] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 08:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 08:58:01.388790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:01.390294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:01.416418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:01.448744 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfa005-ef18-46db-a680-533d69cb46d3 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:01.452403 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f66a6-620b-42ba-9597-3c9be5893197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:01.453001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f66a6-620b-42ba-9597-3c9be5893197 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:01.453279 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f66a6-620b-42ba-9597-3c9be5893197 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:01.453550 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f66a6-620b-42ba-9597-3c9be5893197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:01.463455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f66a6-620b-42ba-9597-3c9be5893197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:01.464879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f66a6-620b-42ba-9597-3c9be5893197 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:01.474181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1399/2782] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:58:01.734113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a712804-b7c5-477d-818e-7147edd8894a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:58:01.735617 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-0a712804-b7c5-477d-818e-7147edd8894a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:58:01.748119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a712804-b7c5-477d-818e-7147edd8894a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:58:01.752769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:58:01.753400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:58:01.753678 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:58:01.753986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] PATH_INFO: `/v3/projects/d2ebd9326b1e41b7a4a5a86f8b770c0b` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:58:01.755131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ebd9326b1e41b7a4a5a86f8b770c0b)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:58:01.757038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:58:01.802401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ebd9326b1e41b7a4a5a86f8b770c0b', 'request_id': 'req-6dba6c0d-12b7-4df3-8362-8c16e6286d03'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:01.809147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ebd9326b1e41b7a4a5a86f8b770c0b', 'request_id': 'req-6dba6c0d-12b7-4df3-8362-8c16e6286d03'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:01.814671 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ebd9326b1e41b7a4a5a86f8b770c0b', 'request_id': 'req-6dba6c0d-12b7-4df3-8362-8c16e6286d03'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:01.827059 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] The token cache is being invalidate because project d2ebd9326b1e41b7a4a5a86f8b770c0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:58:01.827400 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:01.828003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:58:01.829460 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1384/2783] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:58:01 2022] DELETE /identity/v3/projects/d2ebd9326b1e41b7a4a5a86f8b770c0b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:58:11.470094 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=16 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:58:11.824065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:59:23.287157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f66a6-620b-42ba-9597-3c9be5893197 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:59:23.296113 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f66a6-620b-42ba-9597-3c9be5893197 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:59:23.346442 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0927cf2-937e-4f80-82e2-57c7686289df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:59:23.346962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0927cf2-937e-4f80-82e2-57c7686289df service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:59:23.347264 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0927cf2-937e-4f80-82e2-57c7686289df service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:59:23.347533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0927cf2-937e-4f80-82e2-57c7686289df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:59:23.349262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b0927cf2-937e-4f80-82e2-57c7686289df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:59:23.411272 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0927cf2-937e-4f80-82e2-57c7686289df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:59:23.412695 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0927cf2-937e-4f80-82e2-57c7686289df service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:59:23.422144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1400/2784] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:59:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:59:26.614679 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:59:26.616213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:59:26.708499 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba6c0d-12b7-4df3-8362-8c16e6286d03 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:59:26.714151 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25cee7-42bd-4c8a-b5f0-c5db73caa42a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:59:26.714873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25cee7-42bd-4c8a-b5f0-c5db73caa42a service placement] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:59:26.715248 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25cee7-42bd-4c8a-b5f0-c5db73caa42a service placement] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:59:26.715625 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25cee7-42bd-4c8a-b5f0-c5db73caa42a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:59:26.717532 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1d25cee7-42bd-4c8a-b5f0-c5db73caa42a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:59:26.731162 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25cee7-42bd-4c8a-b5f0-c5db73caa42a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:59:26.732784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25cee7-42bd-4c8a-b5f0-c5db73caa42a service placement] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:59:26.744468 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1385/2785] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:59:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 08:59:29.837757 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0927cf2-937e-4f80-82e2-57c7686289df service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:59:29.839293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-b0927cf2-937e-4f80-82e2-57c7686289df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:59:29.852859 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0927cf2-937e-4f80-82e2-57c7686289df service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:59:29.857448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:59:29.858106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:59:29.858388 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:59:29.858690 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] PATH_INFO: `/v3/users/f928852d90b94ac5979e4a7806896b7a` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:59:29.873185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f928852d90b94ac5979e4a7806896b7a)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:59:29.875783 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:59:29.919999 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f928852d90b94ac5979e4a7806896b7a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:29.920678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f928852d90b94ac5979e4a7806896b7a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:29.927946 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f928852d90b94ac5979e4a7806896b7a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:29.928632 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f928852d90b94ac5979e4a7806896b7a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:29.941194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f928852d90b94ac5979e4a7806896b7a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:29.953925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1401/2786] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:59:29 2022] DELETE /identity/v3/users/f928852d90b94ac5979e4a7806896b7a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:59:30.120632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25cee7-42bd-4c8a-b5f0-c5db73caa42a service placement] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:59:30.131769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25cee7-42bd-4c8a-b5f0-c5db73caa42a service placement] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:59:30.197728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:59:30.198383 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:59:30.198682 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:59:30.199079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] PATH_INFO: `/v3/projects/1104ee1508e24a2b87820a8b0fafd0e1` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:59:30.210712 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1104ee1508e24a2b87820a8b0fafd0e1)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:59:30.213003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:59:30.260409 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1104ee1508e24a2b87820a8b0fafd0e1', 'request_id': 'req-406278a8-80a6-4ea0-a272-35a2a459af1e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.269089 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1104ee1508e24a2b87820a8b0fafd0e1', 'request_id': 'req-406278a8-80a6-4ea0-a272-35a2a459af1e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.275528 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1104ee1508e24a2b87820a8b0fafd0e1', 'request_id': 'req-406278a8-80a6-4ea0-a272-35a2a459af1e'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.287688 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] The token cache is being invalidate because project 1104ee1508e24a2b87820a8b0fafd0e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:59:30.288000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.288623 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.290188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1386/2787] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:59:30 2022] DELETE /identity/v3/projects/1104ee1508e24a2b87820a8b0fafd0e1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:59:30.294092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:59:30.295516 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:59:30.309140 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3ddcbd-fef8-483b-9ec7-9d7413c6163b admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:59:30.317401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:59:30.367011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:59:30.367829 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:59:30.368148 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:59:30.368433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] PATH_INFO: `/v3/users/6eb4145e013f49e9a37beeac46009a1b` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:59:30.382660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eb4145e013f49e9a37beeac46009a1b)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:59:30.384645 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:59:30.429093 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eb4145e013f49e9a37beeac46009a1b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.429778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eb4145e013f49e9a37beeac46009a1b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.436683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eb4145e013f49e9a37beeac46009a1b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.437317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eb4145e013f49e9a37beeac46009a1b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.449659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eb4145e013f49e9a37beeac46009a1b'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.462863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1402/2788] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 08:59:30 2022] DELETE /identity/v3/users/6eb4145e013f49e9a37beeac46009a1b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:59:30.639668 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:59:30.641188 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:59:30.654271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406278a8-80a6-4ea0-a272-35a2a459af1e admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:59:30.662654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:59:30.714119 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:59:30.714749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:59:30.715028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:59:30.715303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] PATH_INFO: `/v3/projects/e48a362e41a844ab80e24925dbf53993` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:59:30.725340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e48a362e41a844ab80e24925dbf53993)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:59:30.727282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:59:30.771994 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e48a362e41a844ab80e24925dbf53993', 'request_id': 'req-b2282cac-4809-41f9-920b-513477e69ab1'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.778303 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e48a362e41a844ab80e24925dbf53993', 'request_id': 'req-b2282cac-4809-41f9-920b-513477e69ab1'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.784472 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e48a362e41a844ab80e24925dbf53993', 'request_id': 'req-b2282cac-4809-41f9-920b-513477e69ab1'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.796347 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] The token cache is being invalidate because project e48a362e41a844ab80e24925dbf53993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 08:59:30.796635 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.797818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 08:59:30.799252 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1387/2789] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 08:59:30 2022] DELETE /identity/v3/projects/e48a362e41a844ab80e24925dbf53993 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 08:59:40.459080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:59:40.793343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 08:59:58.588147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 08:59:58.589574 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:59:58.658684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94212c-b4c0-443f-ac75-01191a8bcb05 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 08:59:58.717746 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123020de-6507-4972-8bae-ed0e8895d4d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 08:59:58.718359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123020de-6507-4972-8bae-ed0e8895d4d1 service placement] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 08:59:58.718639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123020de-6507-4972-8bae-ed0e8895d4d1 service placement] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 08:59:58.718912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123020de-6507-4972-8bae-ed0e8895d4d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 08:59:58.720637 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-123020de-6507-4972-8bae-ed0e8895d4d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 08:59:58.731710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123020de-6507-4972-8bae-ed0e8895d4d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 08:59:58.733291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123020de-6507-4972-8bae-ed0e8895d4d1 service placement] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 08:59:58.743284 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1403/2790] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 08:59:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:08.738967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=58 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:40.136020 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:40.137560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:40.195909 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2282cac-4809-41f9-920b-513477e69ab1 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:40.199680 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eebc68-04b4-4a9f-94f2-feef1bb1483a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:40.200313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eebc68-04b4-4a9f-94f2-feef1bb1483a service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:40.200589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eebc68-04b4-4a9f-94f2-feef1bb1483a service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:40.200859 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eebc68-04b4-4a9f-94f2-feef1bb1483a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:40.202634 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-21eebc68-04b4-4a9f-94f2-feef1bb1483a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:40.274967 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eebc68-04b4-4a9f-94f2-feef1bb1483a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:40.276405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eebc68-04b4-4a9f-94f2-feef1bb1483a service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:40.287272 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1388/2791] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:41.470176 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123020de-6507-4972-8bae-ed0e8895d4d1 service placement] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:00:41.471565 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-123020de-6507-4972-8bae-ed0e8895d4d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:41.482584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123020de-6507-4972-8bae-ed0e8895d4d1 service placement] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:00:41.486750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a3a3d-16fe-4576-bb9e-421badabb573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:00:41.487339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a3a3d-16fe-4576-bb9e-421badabb573 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:00:41.487614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a3a3d-16fe-4576-bb9e-421badabb573 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:00:41.487912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a3a3d-16fe-4576-bb9e-421badabb573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:00:41.489556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-5e8a3a3d-16fe-4576-bb9e-421badabb573 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:00:41.505577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a3a3d-16fe-4576-bb9e-421badabb573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:00:41.507051 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a3a3d-16fe-4576-bb9e-421badabb573 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:00:41.518959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1404/2792] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:00:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:00:50.282370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=54 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:00:51.513356 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=6 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:02:00.404822 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eebc68-04b4-4a9f-94f2-feef1bb1483a service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:00.407048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-21eebc68-04b4-4a9f-94f2-feef1bb1483a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:00.423098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eebc68-04b4-4a9f-94f2-feef1bb1483a service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:00.428895 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080191e0-275c-4968-a62c-b02d541521f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:00.430116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080191e0-275c-4968-a62c-b02d541521f0 service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:00.430116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080191e0-275c-4968-a62c-b02d541521f0 service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:00.430116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080191e0-275c-4968-a62c-b02d541521f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:00.432022 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-080191e0-275c-4968-a62c-b02d541521f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:00.448275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080191e0-275c-4968-a62c-b02d541521f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:00.449995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080191e0-275c-4968-a62c-b02d541521f0 service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:00.462706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1389/2793] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:02:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:02:04.526047 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e56dd-c3ed-4d5f-988e-708199a63c9f None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:04.526571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e56dd-c3ed-4d5f-988e-708199a63c9f None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:04.526571 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e56dd-c3ed-4d5f-988e-708199a63c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:04.545756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-a02e56dd-c3ed-4d5f-988e-708199a63c9f None None] MFA Rules not processed for user `c710350e02634e84b3d4700ae5fa6925`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:02:04.546955 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a02e56dd-c3ed-4d5f-988e-708199a63c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:04.600361 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1405/2794] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:02:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:02:04.908467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080191e0-275c-4968-a62c-b02d541521f0 service nova] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:04.918475 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080191e0-275c-4968-a62c-b02d541521f0 service nova] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:04.922101 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01eda9-4dfc-445a-8bbb-58a348655aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:04.922692 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01eda9-4dfc-445a-8bbb-58a348655aef service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:04.922971 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01eda9-4dfc-445a-8bbb-58a348655aef service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:04.923244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01eda9-4dfc-445a-8bbb-58a348655aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:04.924973 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ef01eda9-4dfc-445a-8bbb-58a348655aef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:04.935064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01eda9-4dfc-445a-8bbb-58a348655aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:04.936485 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01eda9-4dfc-445a-8bbb-58a348655aef service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:04.944414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1390/2795] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:02:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:02:05.264209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e56dd-c3ed-4d5f-988e-708199a63c9f None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:05.265389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a02e56dd-c3ed-4d5f-988e-708199a63c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:05.280794 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e56dd-c3ed-4d5f-988e-708199a63c9f None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:05.286006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fad661-913e-4646-80d3-b0b9ba1c3029 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:05.286543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fad661-913e-4646-80d3-b0b9ba1c3029 service placement] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:05.286870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fad661-913e-4646-80d3-b0b9ba1c3029 service placement] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:05.287151 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fad661-913e-4646-80d3-b0b9ba1c3029 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:05.288836 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-69fad661-913e-4646-80d3-b0b9ba1c3029 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:05.300683 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fad661-913e-4646-80d3-b0b9ba1c3029 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:05.305943 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fad661-913e-4646-80d3-b0b9ba1c3029 service placement] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:05.317253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1406/2796] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:02:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:02:07.164666 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01eda9-4dfc-445a-8bbb-58a348655aef service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:07.175285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01eda9-4dfc-445a-8bbb-58a348655aef service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:07.179399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3b797-ebd8-456f-911d-864ccc190530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:07.180213 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3b797-ebd8-456f-911d-864ccc190530 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:07.180494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3b797-ebd8-456f-911d-864ccc190530 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:07.180769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db3b797-ebd8-456f-911d-864ccc190530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:07.182581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4db3b797-ebd8-456f-911d-864ccc190530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:07.195051 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3b797-ebd8-456f-911d-864ccc190530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:07.197049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db3b797-ebd8-456f-911d-864ccc190530 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:07.206765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1391/2797] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:02:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:02:14.729199 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fad661-913e-4646-80d3-b0b9ba1c3029 service placement] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:02:14.743079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fad661-913e-4646-80d3-b0b9ba1c3029 service placement] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:02:14.748445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7786e-d662-456b-8ea9-5d6219c72ec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:02:14.749045 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7786e-d662-456b-8ea9-5d6219c72ec5 service nova] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:02:14.749322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7786e-d662-456b-8ea9-5d6219c72ec5 service nova] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:02:14.749591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7786e-d662-456b-8ea9-5d6219c72ec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:02:14.751322 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-08e7786e-d662-456b-8ea9-5d6219c72ec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:02:14.764494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7786e-d662-456b-8ea9-5d6219c72ec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:02:14.766115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7786e-d662-456b-8ea9-5d6219c72ec5 service nova] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:02:14.776967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1407/2798] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:02:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:02:17.202386 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=18 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:02:24.772301 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=37 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:22.855783 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3b797-ebd8-456f-911d-864ccc190530 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:22.857360 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4db3b797-ebd8-456f-911d-864ccc190530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:22.873038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db3b797-ebd8-456f-911d-864ccc190530 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:22.878513 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f09032-60fb-4e59-99e9-b003555e3b1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee8a19f4d9704408a8b737095e5885f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:22.879102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f09032-60fb-4e59-99e9-b003555e3b1d service nova] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:22.879382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f09032-60fb-4e59-99e9-b003555e3b1d service nova] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:22.879653 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f09032-60fb-4e59-99e9-b003555e3b1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:22.881476 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-27f09032-60fb-4e59-99e9-b003555e3b1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:22.891437 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f09032-60fb-4e59-99e9-b003555e3b1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:22.893009 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f09032-60fb-4e59-99e9-b003555e3b1d service nova] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:22.902442 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1392/2799] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:32.898465 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=6 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:53.394725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7786e-d662-456b-8ea9-5d6219c72ec5 service nova] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:53.413622 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7786e-d662-456b-8ea9-5d6219c72ec5 service nova] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:53.417187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f16581-2ff3-4268-a66c-a98e6664d258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:53.417806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f16581-2ff3-4268-a66c-a98e6664d258 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:53.418085 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f16581-2ff3-4268-a66c-a98e6664d258 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:53.418354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f16581-2ff3-4268-a66c-a98e6664d258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:53.420042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-73f16581-2ff3-4268-a66c-a98e6664d258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:53.531273 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f16581-2ff3-4268-a66c-a98e6664d258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:53.532739 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f16581-2ff3-4268-a66c-a98e6664d258 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:53.551287 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1408/2800] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:56.543651 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f09032-60fb-4e59-99e9-b003555e3b1d service nova] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:56.586191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f09032-60fb-4e59-99e9-b003555e3b1d service nova] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:56.593265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46857745-9da5-4f1c-840f-042175bac3d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:56.593906 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46857745-9da5-4f1c-840f-042175bac3d4 service placement] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:56.594192 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46857745-9da5-4f1c-840f-042175bac3d4 service placement] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:56.594483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46857745-9da5-4f1c-840f-042175bac3d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:56.596582 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-46857745-9da5-4f1c-840f-042175bac3d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:56.604726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f16581-2ff3-4268-a66c-a98e6664d258 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:56.606785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-73f16581-2ff3-4268-a66c-a98e6664d258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:56.623433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f16581-2ff3-4268-a66c-a98e6664d258 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:56.635468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfeb70b-14fe-4e10-9c50-7caf4037f631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:56.635794 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46857745-9da5-4f1c-840f-042175bac3d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:56.636583 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeb70b-14fe-4e10-9c50-7caf4037f631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:56.636927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeb70b-14fe-4e10-9c50-7caf4037f631 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:56.637408 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeb70b-14fe-4e10-9c50-7caf4037f631 admin admin] PATH_INFO: `/v3/users/c710350e02634e84b3d4700ae5fa6925` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:56.637799 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46857745-9da5-4f1c-840f-042175bac3d4 service placement] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:56.638860 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfeb70b-14fe-4e10-9c50-7caf4037f631 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c710350e02634e84b3d4700ae5fa6925)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:56.640840 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfeb70b-14fe-4e10-9c50-7caf4037f631 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:56.649436 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1393/2801] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:56.708775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7bfeb70b-14fe-4e10-9c50-7caf4037f631 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c710350e02634e84b3d4700ae5fa6925'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:56.709521 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7bfeb70b-14fe-4e10-9c50-7caf4037f631 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c710350e02634e84b3d4700ae5fa6925'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:56.717748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7bfeb70b-14fe-4e10-9c50-7caf4037f631 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c710350e02634e84b3d4700ae5fa6925'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:56.718453 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7bfeb70b-14fe-4e10-9c50-7caf4037f631 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c710350e02634e84b3d4700ae5fa6925'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:56.731399 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-7bfeb70b-14fe-4e10-9c50-7caf4037f631 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c710350e02634e84b3d4700ae5fa6925'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:56.742381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:56.749209 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1409/2802] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:56 2022] DELETE /identity/v3/users/c710350e02634e84b3d4700ae5fa6925 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:56.921127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46857745-9da5-4f1c-840f-042175bac3d4 service placement] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:56.938030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46857745-9da5-4f1c-840f-042175bac3d4 service placement] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:56.994911 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3ea2b-c0f5-4dd2-854c-184316eb477b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:56.995556 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3ea2b-c0f5-4dd2-854c-184316eb477b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:56.995835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3ea2b-c0f5-4dd2-854c-184316eb477b admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:56.996102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3ea2b-c0f5-4dd2-854c-184316eb477b admin admin] PATH_INFO: `/v3/projects/4370b750fe184c5eade1378abfa02752` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:56.997212 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3ea2b-c0f5-4dd2-854c-184316eb477b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4370b750fe184c5eade1378abfa02752)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:56.999141 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3ea2b-c0f5-4dd2-854c-184316eb477b admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.045263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2de3ea2b-c0f5-4dd2-854c-184316eb477b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4370b750fe184c5eade1378abfa02752', 'request_id': 'req-2de3ea2b-c0f5-4dd2-854c-184316eb477b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.051180 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2de3ea2b-c0f5-4dd2-854c-184316eb477b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4370b750fe184c5eade1378abfa02752', 'request_id': 'req-2de3ea2b-c0f5-4dd2-854c-184316eb477b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.056558 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2de3ea2b-c0f5-4dd2-854c-184316eb477b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4370b750fe184c5eade1378abfa02752', 'request_id': 'req-2de3ea2b-c0f5-4dd2-854c-184316eb477b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.068700 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2de3ea2b-c0f5-4dd2-854c-184316eb477b admin admin] The token cache is being invalidate because project 4370b750fe184c5eade1378abfa02752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:57.068985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2de3ea2b-c0f5-4dd2-854c-184316eb477b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.069596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-2de3ea2b-c0f5-4dd2-854c-184316eb477b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:57.071045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1394/2803] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:56 2022] DELETE /identity/v3/projects/4370b750fe184c5eade1378abfa02752 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:57.075833 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121bd70-7f94-41fb-9287-20ecebd9dddc None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.076119 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121bd70-7f94-41fb-9287-20ecebd9dddc None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.076395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121bd70-7f94-41fb-9287-20ecebd9dddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.133807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-2121bd70-7f94-41fb-9287-20ecebd9dddc None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:57.134939 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2121bd70-7f94-41fb-9287-20ecebd9dddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:57.192144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1410/2804] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:57.201658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837611c3-d12d-4318-a3d0-0aaa5855ad7c None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.201985 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837611c3-d12d-4318-a3d0-0aaa5855ad7c None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.202268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837611c3-d12d-4318-a3d0-0aaa5855ad7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.222611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-837611c3-d12d-4318-a3d0-0aaa5855ad7c None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:57.223823 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-837611c3-d12d-4318-a3d0-0aaa5855ad7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:57.231681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1395/2805] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:57.235979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121bd70-7f94-41fb-9287-20ecebd9dddc None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.246082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121bd70-7f94-41fb-9287-20ecebd9dddc None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.250691 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390dc56-ee28-47c0-acc4-cbf57d33b4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.251287 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390dc56-ee28-47c0-acc4-cbf57d33b4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.251572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390dc56-ee28-47c0-acc4-cbf57d33b4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.251847 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390dc56-ee28-47c0-acc4-cbf57d33b4e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.252491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390dc56-ee28-47c0-acc4-cbf57d33b4e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:03:57.252798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390dc56-ee28-47c0-acc4-cbf57d33b4e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.254447 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390dc56-ee28-47c0-acc4-cbf57d33b4e5 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.266837 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1411/2806] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 09:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:03:57.271022 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837611c3-d12d-4318-a3d0-0aaa5855ad7c None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.280054 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837611c3-d12d-4318-a3d0-0aaa5855ad7c None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.284713 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b948273-ed67-4bba-9052-92d16cabc82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.285301 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b948273-ed67-4bba-9052-92d16cabc82c admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.285580 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b948273-ed67-4bba-9052-92d16cabc82c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.285882 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b948273-ed67-4bba-9052-92d16cabc82c admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.286510 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b948273-ed67-4bba-9052-92d16cabc82c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.288431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b948273-ed67-4bba-9052-92d16cabc82c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.315791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1396/2807] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:57 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:57.320418 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390dc56-ee28-47c0-acc4-cbf57d33b4e5 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.330283 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390dc56-ee28-47c0-acc4-cbf57d33b4e5 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.386201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200748c9-b8ba-4f00-bbe5-f979a443574d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.386757 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200748c9-b8ba-4f00-bbe5-f979a443574d admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.387071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200748c9-b8ba-4f00-bbe5-f979a443574d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.387536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200748c9-b8ba-4f00-bbe5-f979a443574d admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.388709 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200748c9-b8ba-4f00-bbe5-f979a443574d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.392566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200748c9-b8ba-4f00-bbe5-f979a443574d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.419792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1412/2808] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:57 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:57.423949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b948273-ed67-4bba-9052-92d16cabc82c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.434698 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b948273-ed67-4bba-9052-92d16cabc82c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.439285 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185146e-9140-4e06-ae00-ededbfb3fcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.439878 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185146e-9140-4e06-ae00-ededbfb3fcbc admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.440159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185146e-9140-4e06-ae00-ededbfb3fcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.440434 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185146e-9140-4e06-ae00-ededbfb3fcbc admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.441069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185146e-9140-4e06-ae00-ededbfb3fcbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.443361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185146e-9140-4e06-ae00-ededbfb3fcbc admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.445343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:57.455056 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1397/2809] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:57.458435 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200748c9-b8ba-4f00-bbe5-f979a443574d admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.468476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200748c9-b8ba-4f00-bbe5-f979a443574d admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.473055 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8055de7-9dc1-4331-8f43-c23e24c2bc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.473662 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8055de7-9dc1-4331-8f43-c23e24c2bc32 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.473967 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8055de7-9dc1-4331-8f43-c23e24c2bc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.474237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8055de7-9dc1-4331-8f43-c23e24c2bc32 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.474871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8055de7-9dc1-4331-8f43-c23e24c2bc32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.476436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8055de7-9dc1-4331-8f43-c23e24c2bc32 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.486818 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1413/2810] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:57.491062 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185146e-9140-4e06-ae00-ededbfb3fcbc admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.501225 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185146e-9140-4e06-ae00-ededbfb3fcbc admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.506498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05c433-9e75-4bb8-94e9-1052e879f26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.507148 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05c433-9e75-4bb8-94e9-1052e879f26c admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.507432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05c433-9e75-4bb8-94e9-1052e879f26c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.507723 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05c433-9e75-4bb8-94e9-1052e879f26c admin admin] PATH_INFO: `/v3/projects/85cf7d4cc6444922961ad140d6716d5c/users/26771aa8daa34c8cb4490eab71a4b7ba/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.522022 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05c433-9e75-4bb8-94e9-1052e879f26c admin admin] RBAC: Authorizing `identity:create_grant(project_id=85cf7d4cc6444922961ad140d6716d5c, user_id=26771aa8daa34c8cb4490eab71a4b7ba, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.524647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05c433-9e75-4bb8-94e9-1052e879f26c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.538485 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1398/2811] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:57 2022] PUT /identity/v3/projects/85cf7d4cc6444922961ad140d6716d5c/users/26771aa8daa34c8cb4490eab71a4b7ba/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:57.548682 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c9957-ea8b-4649-a2eb-91a2b273804d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.548971 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c9957-ea8b-4649-a2eb-91a2b273804d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.549246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c9957-ea8b-4649-a2eb-91a2b273804d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.576691 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-c77c9957-ea8b-4649-a2eb-91a2b273804d None None] MFA Rules not processed for user `26771aa8daa34c8cb4490eab71a4b7ba`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:57.577939 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c77c9957-ea8b-4649-a2eb-91a2b273804d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:57.597306 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:57.643001 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1414/2812] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:03:57 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:57.647495 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05c433-9e75-4bb8-94e9-1052e879f26c admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.709309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05c433-9e75-4bb8-94e9-1052e879f26c admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.714261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f3020-1288-412f-bbd0-9211f8dc824d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.714857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f3020-1288-412f-bbd0-9211f8dc824d admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.715136 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f3020-1288-412f-bbd0-9211f8dc824d admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.715405 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f3020-1288-412f-bbd0-9211f8dc824d admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.716020 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f3020-1288-412f-bbd0-9211f8dc824d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.717919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f3020-1288-412f-bbd0-9211f8dc824d admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.744320 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1399/2813] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:57 2022] POST /identity/v3/projects => generated 405 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:57.748594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c9957-ea8b-4649-a2eb-91a2b273804d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.759761 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c9957-ea8b-4649-a2eb-91a2b273804d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.818147 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67944229-7bf2-43df-8da8-aa3b51e3be27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.818760 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67944229-7bf2-43df-8da8-aa3b51e3be27 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.819043 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67944229-7bf2-43df-8da8-aa3b51e3be27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.819316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67944229-7bf2-43df-8da8-aa3b51e3be27 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.819949 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67944229-7bf2-43df-8da8-aa3b51e3be27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.821902 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67944229-7bf2-43df-8da8-aa3b51e3be27 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.843368 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1415/2814] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:57 2022] POST /identity/v3/users => generated 309 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:57.847526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f3020-1288-412f-bbd0-9211f8dc824d admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.857675 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f3020-1288-412f-bbd0-9211f8dc824d admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.862663 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e5f9-ad8e-4305-b383-44aa9aabfacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.863266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e5f9-ad8e-4305-b383-44aa9aabfacb admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.863544 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e5f9-ad8e-4305-b383-44aa9aabfacb admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.863814 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e5f9-ad8e-4305-b383-44aa9aabfacb admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.864958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39e5f9-ad8e-4305-b383-44aa9aabfacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.866566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39e5f9-ad8e-4305-b383-44aa9aabfacb admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.876810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1400/2815] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:57.881428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67944229-7bf2-43df-8da8-aa3b51e3be27 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.891311 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67944229-7bf2-43df-8da8-aa3b51e3be27 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.896005 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ba2e1-3a44-44ec-a8cb-62b090e7adda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.896593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ba2e1-3a44-44ec-a8cb-62b090e7adda admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.896875 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ba2e1-3a44-44ec-a8cb-62b090e7adda admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.897152 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ba2e1-3a44-44ec-a8cb-62b090e7adda admin admin] PATH_INFO: `/v3/projects/d06af86f6af5466792c06b01f6d0f6f3/users/f45f6e00df194e24a6827fb238238ba0/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.911598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ba2e1-3a44-44ec-a8cb-62b090e7adda admin admin] RBAC: Authorizing `identity:create_grant(project_id=d06af86f6af5466792c06b01f6d0f6f3, user_id=f45f6e00df194e24a6827fb238238ba0, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:57.914410 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ba2e1-3a44-44ec-a8cb-62b090e7adda admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:57.927659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1416/2816] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:57 2022] PUT /identity/v3/projects/d06af86f6af5466792c06b01f6d0f6f3/users/f45f6e00df194e24a6827fb238238ba0/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:57.931771 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e5f9-ad8e-4305-b383-44aa9aabfacb admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:57.941647 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e5f9-ad8e-4305-b383-44aa9aabfacb admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:57.996643 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15227e-e73e-49a0-8eee-633f92091f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:57.997275 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15227e-e73e-49a0-8eee-633f92091f2a admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:57.997555 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15227e-e73e-49a0-8eee-633f92091f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:57.997873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15227e-e73e-49a0-8eee-633f92091f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:57.998515 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15227e-e73e-49a0-8eee-633f92091f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:58.000136 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15227e-e73e-49a0-8eee-633f92091f2a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:58.010156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1401/2817] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:03:58.014456 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ba2e1-3a44-44ec-a8cb-62b090e7adda admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:58.024854 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ba2e1-3a44-44ec-a8cb-62b090e7adda admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:58.029510 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cfecd2-81c1-4010-9286-cfd827410b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:58.030188 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfecd2-81c1-4010-9286-cfd827410b08 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.030472 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfecd2-81c1-4010-9286-cfd827410b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.030748 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfecd2-81c1-4010-9286-cfd827410b08 admin admin] PATH_INFO: `/v3/projects/d06af86f6af5466792c06b01f6d0f6f3/users/f45f6e00df194e24a6827fb238238ba0/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.032655 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cfecd2-81c1-4010-9286-cfd827410b08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d06af86f6af5466792c06b01f6d0f6f3, user_id=f45f6e00df194e24a6827fb238238ba0, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:58.035367 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cfecd2-81c1-4010-9286-cfd827410b08 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:58.045634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1417/2818] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:03:58 2022] PUT /identity/v3/projects/d06af86f6af5466792c06b01f6d0f6f3/users/f45f6e00df194e24a6827fb238238ba0/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:58.057371 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1510863-56ed-44cb-a334-3edd58e8efdb None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.057658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1510863-56ed-44cb-a334-3edd58e8efdb None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.057970 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1510863-56ed-44cb-a334-3edd58e8efdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.085997 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c1510863-56ed-44cb-a334-3edd58e8efdb None None] MFA Rules not processed for user `f45f6e00df194e24a6827fb238238ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:58.087498 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c1510863-56ed-44cb-a334-3edd58e8efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:58.119449 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:58.156420 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1402/2819] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:03:58 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:58.167078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cfecd2-81c1-4010-9286-cfd827410b08 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:58.168819 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-80cfecd2-81c1-4010-9286-cfd827410b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:58.226174 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cfecd2-81c1-4010-9286-cfd827410b08 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:58.229909 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2afe9-876f-4836-bd25-83dfcca49e58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:58.230504 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2afe9-876f-4836-bd25-83dfcca49e58 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.230785 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2afe9-876f-4836-bd25-83dfcca49e58 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.231057 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2afe9-876f-4836-bd25-83dfcca49e58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.241289 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2afe9-876f-4836-bd25-83dfcca49e58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:58.243037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2afe9-876f-4836-bd25-83dfcca49e58 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:58.255959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1418/2820] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:58 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:58.398699 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1510863-56ed-44cb-a334-3edd58e8efdb None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:58.459755 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1510863-56ed-44cb-a334-3edd58e8efdb None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:58.464522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:58.465147 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.465431 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.465730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] PATH_INFO: `/v3/users/26771aa8daa34c8cb4490eab71a4b7ba` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.466851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=26771aa8daa34c8cb4490eab71a4b7ba)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:58.468773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:58.514550 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26771aa8daa34c8cb4490eab71a4b7ba'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:58.515196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26771aa8daa34c8cb4490eab71a4b7ba'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:58.526829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26771aa8daa34c8cb4490eab71a4b7ba'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:58.533553 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26771aa8daa34c8cb4490eab71a4b7ba'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:58.534200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26771aa8daa34c8cb4490eab71a4b7ba'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:58.546573 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1403/2821] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:58 2022] DELETE /identity/v3/users/26771aa8daa34c8cb4490eab71a4b7ba => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:58.551120 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f27a60-387d-4c04-b133-b08c1500192a None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.551415 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f27a60-387d-4c04-b133-b08c1500192a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.552334 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f27a60-387d-4c04-b133-b08c1500192a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.570810 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-37f27a60-387d-4c04-b133-b08c1500192a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:58.572003 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-37f27a60-387d-4c04-b133-b08c1500192a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:58.583351 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:58.633667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1419/2822] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:03:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:03:58.643333 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:58.644772 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:58.697398 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:58.701345 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a665e-8dcc-4b10-8887-317bdd16ed4a admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:58.705962 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd823d-6d0a-44fa-945c-175516645ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:58.706705 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd823d-6d0a-44fa-945c-175516645ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:58.707177 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd823d-6d0a-44fa-945c-175516645ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:58.707615 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd823d-6d0a-44fa-945c-175516645ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:58.717578 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd823d-6d0a-44fa-945c-175516645ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:58.718986 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd823d-6d0a-44fa-945c-175516645ea8 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:58.728875 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1404/2823] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:03:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:03:58.994566 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f27a60-387d-4c04-b133-b08c1500192a None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:58.996112 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-37f27a60-387d-4c04-b133-b08c1500192a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:59.009675 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f27a60-387d-4c04-b133-b08c1500192a None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:59.014505 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:59.015142 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:59.015423 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:59.016189 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] PATH_INFO: `/v3/projects/85cf7d4cc6444922961ad140d6716d5c` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:59.017428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85cf7d4cc6444922961ad140d6716d5c)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:59.019445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:59.065363 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85cf7d4cc6444922961ad140d6716d5c', 'request_id': 'req-caec9d48-6504-4023-9745-faa044445909'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.071927 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85cf7d4cc6444922961ad140d6716d5c', 'request_id': 'req-caec9d48-6504-4023-9745-faa044445909'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.077375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85cf7d4cc6444922961ad140d6716d5c', 'request_id': 'req-caec9d48-6504-4023-9745-faa044445909'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.090290 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] The token cache is being invalidate because project 85cf7d4cc6444922961ad140d6716d5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:59.090580 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.091201 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.092667 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1420/2824] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:58 2022] DELETE /identity/v3/projects/85cf7d4cc6444922961ad140d6716d5c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:59.096847 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd823d-6d0a-44fa-945c-175516645ea8 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:59.106417 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd823d-6d0a-44fa-945c-175516645ea8 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:59.163024 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8d185-fef4-47d0-b2bf-15e4988555ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:59.163654 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8d185-fef4-47d0-b2bf-15e4988555ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:59.163939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8d185-fef4-47d0-b2bf-15e4988555ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:59.164216 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8d185-fef4-47d0-b2bf-15e4988555ea admin admin] PATH_INFO: `/v3/users/f45f6e00df194e24a6827fb238238ba0` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:59.165335 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8d185-fef4-47d0-b2bf-15e4988555ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f45f6e00df194e24a6827fb238238ba0)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:59.167338 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8d185-fef4-47d0-b2bf-15e4988555ea admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:59.212354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-41f8d185-fef4-47d0-b2bf-15e4988555ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f45f6e00df194e24a6827fb238238ba0'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.213027 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-41f8d185-fef4-47d0-b2bf-15e4988555ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f45f6e00df194e24a6827fb238238ba0'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.225087 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-41f8d185-fef4-47d0-b2bf-15e4988555ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f45f6e00df194e24a6827fb238238ba0'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.231897 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-41f8d185-fef4-47d0-b2bf-15e4988555ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f45f6e00df194e24a6827fb238238ba0'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.232535 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-41f8d185-fef4-47d0-b2bf-15e4988555ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f45f6e00df194e24a6827fb238238ba0'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.245219 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1405/2825] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 09:03:59 2022] DELETE /identity/v3/users/f45f6e00df194e24a6827fb238238ba0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:03:59.513671 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:59.515145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:59.529488 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec9d48-6504-4023-9745-faa044445909 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:59.586266 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d8521-a804-48eb-8c1b-b75d69fc1120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:59.587206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d8521-a804-48eb-8c1b-b75d69fc1120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:59.587487 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d8521-a804-48eb-8c1b-b75d69fc1120 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:59.587753 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d8521-a804-48eb-8c1b-b75d69fc1120 admin admin] PATH_INFO: `/v3/projects/d06af86f6af5466792c06b01f6d0f6f3` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:59.588870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d8521-a804-48eb-8c1b-b75d69fc1120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d06af86f6af5466792c06b01f6d0f6f3)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:59.590874 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d8521-a804-48eb-8c1b-b75d69fc1120 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:59.634227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:59.638044 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-747d8521-a804-48eb-8c1b-b75d69fc1120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd06af86f6af5466792c06b01f6d0f6f3', 'request_id': 'req-747d8521-a804-48eb-8c1b-b75d69fc1120'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.645168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-747d8521-a804-48eb-8c1b-b75d69fc1120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd06af86f6af5466792c06b01f6d0f6f3', 'request_id': 'req-747d8521-a804-48eb-8c1b-b75d69fc1120'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.650806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-747d8521-a804-48eb-8c1b-b75d69fc1120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd06af86f6af5466792c06b01f6d0f6f3', 'request_id': 'req-747d8521-a804-48eb-8c1b-b75d69fc1120'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.663365 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-747d8521-a804-48eb-8c1b-b75d69fc1120 admin admin] The token cache is being invalidate because project d06af86f6af5466792c06b01f6d0f6f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:03:59.663705 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-747d8521-a804-48eb-8c1b-b75d69fc1120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.664331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-747d8521-a804-48eb-8c1b-b75d69fc1120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:03:59.665889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1421/2826] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 09:03:59 2022] DELETE /identity/v3/projects/d06af86f6af5466792c06b01f6d0f6f3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:03:59.670735 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d593d-3a61-4e34-a215-35f3aebd7bbd None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:59.671047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d593d-3a61-4e34-a215-35f3aebd7bbd None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:59.671322 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d593d-3a61-4e34-a215-35f3aebd7bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:59.690844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-196d593d-3a61-4e34-a215-35f3aebd7bbd None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:59.692021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-196d593d-3a61-4e34-a215-35f3aebd7bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:59.748469 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1406/2827] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:59.756751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c5ead-394c-43e6-b1c8-bd994d10aa97 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:59.756976 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c5ead-394c-43e6-b1c8-bd994d10aa97 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:59.757250 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c5ead-394c-43e6-b1c8-bd994d10aa97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:59.776587 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-966c5ead-394c-43e6-b1c8-bd994d10aa97 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:03:59.777800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-966c5ead-394c-43e6-b1c8-bd994d10aa97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:03:59.786459 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1422/2828] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:03:59.790395 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d593d-3a61-4e34-a215-35f3aebd7bbd None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:59.799773 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d593d-3a61-4e34-a215-35f3aebd7bbd None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:59.804957 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d186f-5908-45c0-b765-1f31c7499e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:59.805542 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d186f-5908-45c0-b765-1f31c7499e9f admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:59.805858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d186f-5908-45c0-b765-1f31c7499e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:59.806131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d186f-5908-45c0-b765-1f31c7499e9f admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:59.806764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d186f-5908-45c0-b765-1f31c7499e9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:03:59.807065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d186f-5908-45c0-b765-1f31c7499e9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:59.808638 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d186f-5908-45c0-b765-1f31c7499e9f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:59.815131 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:03:59.821137 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1407/2829] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 09:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:03:59.825115 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c5ead-394c-43e6-b1c8-bd994d10aa97 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:59.834291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c5ead-394c-43e6-b1c8-bd994d10aa97 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:59.839316 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70feae45-a434-456f-bda7-dc7d505f83c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:59.839904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70feae45-a434-456f-bda7-dc7d505f83c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:59.840182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70feae45-a434-456f-bda7-dc7d505f83c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:59.840451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70feae45-a434-456f-bda7-dc7d505f83c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:59.841081 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70feae45-a434-456f-bda7-dc7d505f83c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:59.843058 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70feae45-a434-456f-bda7-dc7d505f83c9 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:59.870099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1423/2830] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 09:03:59 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:59.873861 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d186f-5908-45c0-b765-1f31c7499e9f admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:59.883384 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d186f-5908-45c0-b765-1f31c7499e9f admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:59.939064 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f53377-eee1-456e-a2bc-7b17e3ccb51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:59.939660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53377-eee1-456e-a2bc-7b17e3ccb51e admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:59.939937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53377-eee1-456e-a2bc-7b17e3ccb51e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:59.940206 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53377-eee1-456e-a2bc-7b17e3ccb51e admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:59.940832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f53377-eee1-456e-a2bc-7b17e3ccb51e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:59.942796 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f53377-eee1-456e-a2bc-7b17e3ccb51e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:59.964241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1408/2831] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 09:03:59 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:03:59.968410 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70feae45-a434-456f-bda7-dc7d505f83c9 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:03:59.979018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70feae45-a434-456f-bda7-dc7d505f83c9 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:03:59.983491 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337b80e-aefb-4ca1-ba42-cb07d81277ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:03:59.984103 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337b80e-aefb-4ca1-ba42-cb07d81277ff admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:03:59.984380 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337b80e-aefb-4ca1-ba42-cb07d81277ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:03:59.984650 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337b80e-aefb-4ca1-ba42-cb07d81277ff admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:03:59.985280 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337b80e-aefb-4ca1-ba42-cb07d81277ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:03:59.986895 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337b80e-aefb-4ca1-ba42-cb07d81277ff admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:03:59.997088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1424/2832] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:00.000843 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f53377-eee1-456e-a2bc-7b17e3ccb51e admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:00.010687 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f53377-eee1-456e-a2bc-7b17e3ccb51e admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:00.015150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447cf15-5051-422c-81cf-3dd4755750b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:00.015761 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447cf15-5051-422c-81cf-3dd4755750b4 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:00.016047 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447cf15-5051-422c-81cf-3dd4755750b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:00.016319 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447cf15-5051-422c-81cf-3dd4755750b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:00.016949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447cf15-5051-422c-81cf-3dd4755750b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:00.018744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447cf15-5051-422c-81cf-3dd4755750b4 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:00.028954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1409/2833] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:00.032960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337b80e-aefb-4ca1-ba42-cb07d81277ff admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:00.043125 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337b80e-aefb-4ca1-ba42-cb07d81277ff admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:00.047625 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432899d4-da9b-417a-b938-db407166ebfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:00.048251 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432899d4-da9b-417a-b938-db407166ebfb admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:00.048528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432899d4-da9b-417a-b938-db407166ebfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:00.048797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432899d4-da9b-417a-b938-db407166ebfb admin admin] PATH_INFO: `/v3/projects/097dc5e2db2e4cfa8d62a2ca37438830/users/bf2b64d9769f4d3cb52be26a17b1e5c7/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:00.063358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432899d4-da9b-417a-b938-db407166ebfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=097dc5e2db2e4cfa8d62a2ca37438830, user_id=bf2b64d9769f4d3cb52be26a17b1e5c7, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:00.066009 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432899d4-da9b-417a-b938-db407166ebfb admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:00.079314 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1425/2834] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:00 2022] PUT /identity/v3/projects/097dc5e2db2e4cfa8d62a2ca37438830/users/bf2b64d9769f4d3cb52be26a17b1e5c7/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:00.089661 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035edf30-e566-414a-9392-ab8045a67bcf None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:00.089975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035edf30-e566-414a-9392-ab8045a67bcf None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:00.090250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035edf30-e566-414a-9392-ab8045a67bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:00.122508 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-035edf30-e566-414a-9392-ab8045a67bcf None None] MFA Rules not processed for user `bf2b64d9769f4d3cb52be26a17b1e5c7`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:00.123750 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-035edf30-e566-414a-9392-ab8045a67bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:00.187200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1410/2835] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:00.191682 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432899d4-da9b-417a-b938-db407166ebfb admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:00.252959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432899d4-da9b-417a-b938-db407166ebfb admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:00.257676 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c22544-a92f-4093-86bd-9e5a1dd4676e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:00.258300 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c22544-a92f-4093-86bd-9e5a1dd4676e admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:00.258577 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c22544-a92f-4093-86bd-9e5a1dd4676e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:00.258868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c22544-a92f-4093-86bd-9e5a1dd4676e admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:00.259512 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c22544-a92f-4093-86bd-9e5a1dd4676e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:00.262456 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c22544-a92f-4093-86bd-9e5a1dd4676e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:00.290774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1426/2836] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 09:04:00 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:04:00.294977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035edf30-e566-414a-9392-ab8045a67bcf None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:00.305737 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035edf30-e566-414a-9392-ab8045a67bcf None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:00.360615 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bea6a-c034-49f5-9439-a9509ba59a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:00.361214 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bea6a-c034-49f5-9439-a9509ba59a81 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:00.361490 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bea6a-c034-49f5-9439-a9509ba59a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:00.361790 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8bea6a-c034-49f5-9439-a9509ba59a81 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:00.362407 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8bea6a-c034-49f5-9439-a9509ba59a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:00.364332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8bea6a-c034-49f5-9439-a9509ba59a81 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:00.375224 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:00.386599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1411/2837] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 09:04:00 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:04:00.390758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c22544-a92f-4093-86bd-9e5a1dd4676e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:00.401555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c22544-a92f-4093-86bd-9e5a1dd4676e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:00.406241 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f87a2-0396-4ff1-9604-5514ed48e66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:00.406846 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f87a2-0396-4ff1-9604-5514ed48e66d admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:00.407117 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f87a2-0396-4ff1-9604-5514ed48e66d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:00.407389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f87a2-0396-4ff1-9604-5514ed48e66d admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:00.408027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f87a2-0396-4ff1-9604-5514ed48e66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:00.409624 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f87a2-0396-4ff1-9604-5514ed48e66d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:00.421303 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1427/2838] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:00.427149 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bea6a-c034-49f5-9439-a9509ba59a81 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:00.436947 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8bea6a-c034-49f5-9439-a9509ba59a81 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:00.441509 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16bfec-a6ca-47e1-b3e4-633b893090ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:00.442189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16bfec-a6ca-47e1-b3e4-633b893090ba admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:00.442471 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16bfec-a6ca-47e1-b3e4-633b893090ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:00.442744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16bfec-a6ca-47e1-b3e4-633b893090ba admin admin] PATH_INFO: `/v3/projects/0e5e69541c56440f81cf64a847a95d20/users/eceeb9f9b87140e9b3d0eabe9a084abf/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:00.458626 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16bfec-a6ca-47e1-b3e4-633b893090ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5e69541c56440f81cf64a847a95d20, user_id=eceeb9f9b87140e9b3d0eabe9a084abf, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:00.461300 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16bfec-a6ca-47e1-b3e4-633b893090ba admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:00.475995 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1412/2839] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:00 2022] PUT /identity/v3/projects/0e5e69541c56440f81cf64a847a95d20/users/eceeb9f9b87140e9b3d0eabe9a084abf/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:00.479894 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f87a2-0396-4ff1-9604-5514ed48e66d admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:00.490628 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f87a2-0396-4ff1-9604-5514ed48e66d admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:00.531536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:00.549627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42702a8c-5ad9-4fa2-8049-ce46f758be0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:00.550263 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42702a8c-5ad9-4fa2-8049-ce46f758be0b admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:00.550551 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42702a8c-5ad9-4fa2-8049-ce46f758be0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:00.550826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42702a8c-5ad9-4fa2-8049-ce46f758be0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:00.551474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42702a8c-5ad9-4fa2-8049-ce46f758be0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:00.553216 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42702a8c-5ad9-4fa2-8049-ce46f758be0b admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:00.562918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1428/2840] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:00.567683 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16bfec-a6ca-47e1-b3e4-633b893090ba admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:00.577996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16bfec-a6ca-47e1-b3e4-633b893090ba admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:00.582575 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bc67a-d722-4cd8-b0c4-5d590f1d9823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:00.583292 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bc67a-d722-4cd8-b0c4-5d590f1d9823 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:00.583566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bc67a-d722-4cd8-b0c4-5d590f1d9823 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:00.583840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bc67a-d722-4cd8-b0c4-5d590f1d9823 admin admin] PATH_INFO: `/v3/projects/0e5e69541c56440f81cf64a847a95d20/users/eceeb9f9b87140e9b3d0eabe9a084abf/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:00.585759 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bc67a-d722-4cd8-b0c4-5d590f1d9823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5e69541c56440f81cf64a847a95d20, user_id=eceeb9f9b87140e9b3d0eabe9a084abf, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:00.588630 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bc67a-d722-4cd8-b0c4-5d590f1d9823 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:00.599066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1413/2841] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:00 2022] PUT /identity/v3/projects/0e5e69541c56440f81cf64a847a95d20/users/eceeb9f9b87140e9b3d0eabe9a084abf/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:00.610649 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fca5a-8107-4b36-80c6-8efe3244b8f6 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:00.610945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fca5a-8107-4b36-80c6-8efe3244b8f6 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:00.611229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fca5a-8107-4b36-80c6-8efe3244b8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:00.640128 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-9e7fca5a-8107-4b36-80c6-8efe3244b8f6 None None] MFA Rules not processed for user `eceeb9f9b87140e9b3d0eabe9a084abf`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:00.641354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9e7fca5a-8107-4b36-80c6-8efe3244b8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:00.709455 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1429/2842] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:00.721092 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bc67a-d722-4cd8-b0c4-5d590f1d9823 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:00.722831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-338bc67a-d722-4cd8-b0c4-5d590f1d9823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:00.780845 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bc67a-d722-4cd8-b0c4-5d590f1d9823 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:00.784596 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9bcc2-5474-412e-84e9-22d45db4a616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:00.785133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9bcc2-5474-412e-84e9-22d45db4a616 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:00.785410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9bcc2-5474-412e-84e9-22d45db4a616 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:00.785681 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9bcc2-5474-412e-84e9-22d45db4a616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:00.795646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9bcc2-5474-412e-84e9-22d45db4a616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:00.797067 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9bcc2-5474-412e-84e9-22d45db4a616 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:00.807949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1414/2843] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:00.887536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fca5a-8107-4b36-80c6-8efe3244b8f6 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:00.897879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fca5a-8107-4b36-80c6-8efe3244b8f6 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:00.901476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ec04a-df69-472d-897a-0580448470dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:00.902106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ec04a-df69-472d-897a-0580448470dc service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:00.902389 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ec04a-df69-472d-897a-0580448470dc service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:00.902662 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ec04a-df69-472d-897a-0580448470dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:00.959140 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ec04a-df69-472d-897a-0580448470dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:00.960574 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ec04a-df69-472d-897a-0580448470dc service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:00.969529 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1430/2844] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:10.793045 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9bcc2-5474-412e-84e9-22d45db4a616 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:10.853370 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9bcc2-5474-412e-84e9-22d45db4a616 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:10.857917 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:10.858481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:10.858757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:10.859028 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] PATH_INFO: `/v3/users/bf2b64d9769f4d3cb52be26a17b1e5c7` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:10.860127 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf2b64d9769f4d3cb52be26a17b1e5c7)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:10.862071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:10.906696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf2b64d9769f4d3cb52be26a17b1e5c7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:10.907326 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf2b64d9769f4d3cb52be26a17b1e5c7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:10.917835 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:10.919029 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf2b64d9769f4d3cb52be26a17b1e5c7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:10.927408 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf2b64d9769f4d3cb52be26a17b1e5c7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:10.928313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf2b64d9769f4d3cb52be26a17b1e5c7'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:10.940182 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1415/2845] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 09:04:10 2022] DELETE /identity/v3/users/bf2b64d9769f4d3cb52be26a17b1e5c7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:10.945162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05b70a-48f1-4413-8325-30d60cff09ee None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:10.945639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05b70a-48f1-4413-8325-30d60cff09ee None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:10.945795 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05b70a-48f1-4413-8325-30d60cff09ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:10.965253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-7a05b70a-48f1-4413-8325-30d60cff09ee None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:10.966485 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7a05b70a-48f1-4413-8325-30d60cff09ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:11.024599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1431/2846] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:11.034306 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:11.035787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:11.091868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6974e87-fc82-4cc0-8d5b-89e3ffe80eeb admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:11.095489 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9acfb-bc60-416b-95da-891cc720a95d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:11.096081 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9acfb-bc60-416b-95da-891cc720a95d service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:11.096362 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9acfb-bc60-416b-95da-891cc720a95d service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:11.096637 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9acfb-bc60-416b-95da-891cc720a95d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:11.105468 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9acfb-bc60-416b-95da-891cc720a95d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:11.106924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9acfb-bc60-416b-95da-891cc720a95d service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:11.116748 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1416/2847] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:11.381349 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05b70a-48f1-4413-8325-30d60cff09ee None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:11.382915 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-7a05b70a-48f1-4413-8325-30d60cff09ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:11.391276 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=100 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:11.397600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05b70a-48f1-4413-8325-30d60cff09ee None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:11.402355 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:11.402991 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:11.403269 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:11.403541 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] PATH_INFO: `/v3/projects/097dc5e2db2e4cfa8d62a2ca37438830` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:11.404679 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=097dc5e2db2e4cfa8d62a2ca37438830)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:11.406616 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:11.458672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '097dc5e2db2e4cfa8d62a2ca37438830', 'request_id': 'req-aa59efca-1226-42a0-bdce-d50a5f69f8c7'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:11.465092 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '097dc5e2db2e4cfa8d62a2ca37438830', 'request_id': 'req-aa59efca-1226-42a0-bdce-d50a5f69f8c7'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:11.470649 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '097dc5e2db2e4cfa8d62a2ca37438830', 'request_id': 'req-aa59efca-1226-42a0-bdce-d50a5f69f8c7'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:11.483175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] The token cache is being invalidate because project 097dc5e2db2e4cfa8d62a2ca37438830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:04:11.483466 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:11.484090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:11.485579 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1432/2848] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 09:04:11 2022] DELETE /identity/v3/projects/097dc5e2db2e4cfa8d62a2ca37438830 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:11.490244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9acfb-bc60-416b-95da-891cc720a95d service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:11.499624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9acfb-bc60-416b-95da-891cc720a95d service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:11.554693 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829d2ea-8238-4f0e-8b0b-9ed1f2a28450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:11.555321 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829d2ea-8238-4f0e-8b0b-9ed1f2a28450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:11.555599 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829d2ea-8238-4f0e-8b0b-9ed1f2a28450 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:11.555872 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829d2ea-8238-4f0e-8b0b-9ed1f2a28450 admin admin] PATH_INFO: `/v3/users/eceeb9f9b87140e9b3d0eabe9a084abf` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:11.556983 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829d2ea-8238-4f0e-8b0b-9ed1f2a28450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eceeb9f9b87140e9b3d0eabe9a084abf)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:11.558935 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829d2ea-8238-4f0e-8b0b-9ed1f2a28450 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:11.604873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c829d2ea-8238-4f0e-8b0b-9ed1f2a28450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eceeb9f9b87140e9b3d0eabe9a084abf'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:11.605507 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c829d2ea-8238-4f0e-8b0b-9ed1f2a28450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eceeb9f9b87140e9b3d0eabe9a084abf'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:11.617577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c829d2ea-8238-4f0e-8b0b-9ed1f2a28450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eceeb9f9b87140e9b3d0eabe9a084abf'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:11.624373 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c829d2ea-8238-4f0e-8b0b-9ed1f2a28450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eceeb9f9b87140e9b3d0eabe9a084abf'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:11.624960 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c829d2ea-8238-4f0e-8b0b-9ed1f2a28450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eceeb9f9b87140e9b3d0eabe9a084abf'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:11.636729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1417/2849] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 09:04:11 2022] DELETE /identity/v3/users/eceeb9f9b87140e9b3d0eabe9a084abf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:11.908489 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:11.910018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:11.926154 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59efca-1226-42a0-bdce-d50a5f69f8c7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:11.983637 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:11.984340 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:11.984523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:11.984793 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f admin admin] PATH_INFO: `/v3/projects/0e5e69541c56440f81cf64a847a95d20` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:11.985922 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e5e69541c56440f81cf64a847a95d20)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:11.987830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.032779 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e5e69541c56440f81cf64a847a95d20', 'request_id': 'req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:12.039285 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e5e69541c56440f81cf64a847a95d20', 'request_id': 'req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:12.044713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e5e69541c56440f81cf64a847a95d20', 'request_id': 'req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:12.056485 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f admin admin] The token cache is being invalidate because project 0e5e69541c56440f81cf64a847a95d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:04:12.056768 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:12.057381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a46f8fd7-ae01-4366-b491-0ca22fdb1f4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:12.058850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1433/2850] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 09:04:11 2022] DELETE /identity/v3/projects/0e5e69541c56440f81cf64a847a95d20 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:12.063639 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b4945-cb39-4352-9037-7cf454909386 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.064102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b4945-cb39-4352-9037-7cf454909386 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.064240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b4945-cb39-4352-9037-7cf454909386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.071810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:12.086293 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-5b4b4945-cb39-4352-9037-7cf454909386 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:12.087787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-5b4b4945-cb39-4352-9037-7cf454909386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:12.145826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1418/2851] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:12.153401 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd90494-8ca7-4c98-a346-8a2607496cdd None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.153629 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd90494-8ca7-4c98-a346-8a2607496cdd None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.153933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd90494-8ca7-4c98-a346-8a2607496cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.173021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-4dd90494-8ca7-4c98-a346-8a2607496cdd None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:12.174236 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4dd90494-8ca7-4c98-a346-8a2607496cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:12.181983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1434/2852] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:12.185944 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b4945-cb39-4352-9037-7cf454909386 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:12.196093 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b4945-cb39-4352-9037-7cf454909386 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:12.200740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b794929-c95c-4061-aa39-34ff57928649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:12.201332 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b794929-c95c-4061-aa39-34ff57928649 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.201610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b794929-c95c-4061-aa39-34ff57928649 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.201929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b794929-c95c-4061-aa39-34ff57928649 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.202569 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b794929-c95c-4061-aa39-34ff57928649 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:04:12.202873 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b794929-c95c-4061-aa39-34ff57928649 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:12.204482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b794929-c95c-4061-aa39-34ff57928649 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.216511 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1419/2853] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 09:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:04:12.220639 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd90494-8ca7-4c98-a346-8a2607496cdd None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:12.229598 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd90494-8ca7-4c98-a346-8a2607496cdd None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:12.234196 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61631c8-09e3-484e-975a-bc275a23e0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:12.234777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61631c8-09e3-484e-975a-bc275a23e0b2 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.235054 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61631c8-09e3-484e-975a-bc275a23e0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.235359 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61631c8-09e3-484e-975a-bc275a23e0b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.235987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61631c8-09e3-484e-975a-bc275a23e0b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:12.237903 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61631c8-09e3-484e-975a-bc275a23e0b2 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.243087 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:12.266046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1435/2854] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 09:04:12 2022] POST /identity/v3/projects => generated 401 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:04:12.270729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b794929-c95c-4061-aa39-34ff57928649 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:12.280602 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b794929-c95c-4061-aa39-34ff57928649 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:12.334597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94dc87-bfd9-4387-bc3f-ee30f80e0218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:12.335199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94dc87-bfd9-4387-bc3f-ee30f80e0218 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.335476 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94dc87-bfd9-4387-bc3f-ee30f80e0218 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.335754 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94dc87-bfd9-4387-bc3f-ee30f80e0218 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.336642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94dc87-bfd9-4387-bc3f-ee30f80e0218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:12.338584 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94dc87-bfd9-4387-bc3f-ee30f80e0218 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.358785 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1420/2855] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 09:04:12 2022] POST /identity/v3/users => generated 301 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:04:12.362822 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61631c8-09e3-484e-975a-bc275a23e0b2 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:12.373362 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61631c8-09e3-484e-975a-bc275a23e0b2 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:12.378007 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077d25f-4503-4c7a-a25b-539b4c1bb29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:12.378600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077d25f-4503-4c7a-a25b-539b4c1bb29e admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.378882 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077d25f-4503-4c7a-a25b-539b4c1bb29e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.379162 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077d25f-4503-4c7a-a25b-539b4c1bb29e admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.380332 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077d25f-4503-4c7a-a25b-539b4c1bb29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:12.381959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077d25f-4503-4c7a-a25b-539b4c1bb29e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.391755 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1436/2856] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:12.396139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94dc87-bfd9-4387-bc3f-ee30f80e0218 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:12.405930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94dc87-bfd9-4387-bc3f-ee30f80e0218 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:12.410459 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca34df4-7c24-4749-8a74-70c2f7dbdf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:12.411078 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca34df4-7c24-4749-8a74-70c2f7dbdf12 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.411357 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca34df4-7c24-4749-8a74-70c2f7dbdf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.411626 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca34df4-7c24-4749-8a74-70c2f7dbdf12 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.412259 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca34df4-7c24-4749-8a74-70c2f7dbdf12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:12.413863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca34df4-7c24-4749-8a74-70c2f7dbdf12 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.424164 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1421/2857] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:12.428143 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077d25f-4503-4c7a-a25b-539b4c1bb29e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:12.438556 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077d25f-4503-4c7a-a25b-539b4c1bb29e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:12.443788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27817433-1ef6-4ae8-b24d-cfb7c260deae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:12.444432 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27817433-1ef6-4ae8-b24d-cfb7c260deae admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.444713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27817433-1ef6-4ae8-b24d-cfb7c260deae admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.444987 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27817433-1ef6-4ae8-b24d-cfb7c260deae admin admin] PATH_INFO: `/v3/projects/11d961e34acf45e29c6c24241ca9d438/users/15d8231d652e450bb0255d05583e4bc9/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.459806 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27817433-1ef6-4ae8-b24d-cfb7c260deae admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d961e34acf45e29c6c24241ca9d438, user_id=15d8231d652e450bb0255d05583e4bc9, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:12.462758 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27817433-1ef6-4ae8-b24d-cfb7c260deae admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.475113 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1437/2858] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:12 2022] PUT /identity/v3/projects/11d961e34acf45e29c6c24241ca9d438/users/15d8231d652e450bb0255d05583e4bc9/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:12.485236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a7330-4d46-46cf-9007-d1b7e377240f None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.485523 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a7330-4d46-46cf-9007-d1b7e377240f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.485821 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a7330-4d46-46cf-9007-d1b7e377240f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.513079 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-c48a7330-4d46-46cf-9007-d1b7e377240f None None] MFA Rules not processed for user `15d8231d652e450bb0255d05583e4bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:12.514306 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c48a7330-4d46-46cf-9007-d1b7e377240f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:12.571862 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:12.577740 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1422/2859] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:12 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:12.582180 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27817433-1ef6-4ae8-b24d-cfb7c260deae admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:12.642312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27817433-1ef6-4ae8-b24d-cfb7c260deae admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:12.647425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbffb4-80ec-430a-9383-6ac20f3d5e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:12.648018 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbffb4-80ec-430a-9383-6ac20f3d5e25 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.648293 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbffb4-80ec-430a-9383-6ac20f3d5e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.648561 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbffb4-80ec-430a-9383-6ac20f3d5e25 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.649193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbffb4-80ec-430a-9383-6ac20f3d5e25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:12.651129 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbffb4-80ec-430a-9383-6ac20f3d5e25 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.677465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1438/2860] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 09:04:12 2022] POST /identity/v3/projects => generated 399 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:04:12.681589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a7330-4d46-46cf-9007-d1b7e377240f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:12.692730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a7330-4d46-46cf-9007-d1b7e377240f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:12.748282 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1661553-e24b-41e6-974b-dee18ee1bb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:12.748891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1661553-e24b-41e6-974b-dee18ee1bb20 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.749172 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1661553-e24b-41e6-974b-dee18ee1bb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.749444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1661553-e24b-41e6-974b-dee18ee1bb20 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.750103 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1661553-e24b-41e6-974b-dee18ee1bb20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:12.752011 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1661553-e24b-41e6-974b-dee18ee1bb20 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.772829 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1423/2861] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 09:04:12 2022] POST /identity/v3/users => generated 306 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:04:12.776839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbffb4-80ec-430a-9383-6ac20f3d5e25 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:12.787194 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbffb4-80ec-430a-9383-6ac20f3d5e25 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:12.791788 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6417b-fd6d-40f4-aac0-9c8db1b34e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:12.792386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6417b-fd6d-40f4-aac0-9c8db1b34e76 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.792670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6417b-fd6d-40f4-aac0-9c8db1b34e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.792962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6417b-fd6d-40f4-aac0-9c8db1b34e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.793591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6417b-fd6d-40f4-aac0-9c8db1b34e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:12.795233 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6417b-fd6d-40f4-aac0-9c8db1b34e76 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.806222 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1439/2862] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:12.810050 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1661553-e24b-41e6-974b-dee18ee1bb20 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:12.820354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1661553-e24b-41e6-974b-dee18ee1bb20 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:12.824908 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70fdd4-1533-4c28-bfd0-39b374e6c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:12.825538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70fdd4-1533-4c28-bfd0-39b374e6c037 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.825851 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70fdd4-1533-4c28-bfd0-39b374e6c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.826124 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70fdd4-1533-4c28-bfd0-39b374e6c037 admin admin] PATH_INFO: `/v3/projects/b8f2ae169328402b8b7daf560548f3ac/users/d61464d586df46cdb7b14f47345c4d26/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.840684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70fdd4-1533-4c28-bfd0-39b374e6c037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f2ae169328402b8b7daf560548f3ac, user_id=d61464d586df46cdb7b14f47345c4d26, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:12.843361 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb70fdd4-1533-4c28-bfd0-39b374e6c037 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.855662 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1424/2863] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:12 2022] PUT /identity/v3/projects/b8f2ae169328402b8b7daf560548f3ac/users/d61464d586df46cdb7b14f47345c4d26/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:12.859451 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6417b-fd6d-40f4-aac0-9c8db1b34e76 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:12.870689 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6417b-fd6d-40f4-aac0-9c8db1b34e76 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:12.927899 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2747f7-cb6c-463d-99dd-3137fe95c2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:12.928555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2747f7-cb6c-463d-99dd-3137fe95c2be admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.928841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2747f7-cb6c-463d-99dd-3137fe95c2be admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.929118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2747f7-cb6c-463d-99dd-3137fe95c2be admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.929800 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2747f7-cb6c-463d-99dd-3137fe95c2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:12.931405 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2747f7-cb6c-463d-99dd-3137fe95c2be admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.940972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1440/2864] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:12.945122 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70fdd4-1533-4c28-bfd0-39b374e6c037 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:12.955102 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70fdd4-1533-4c28-bfd0-39b374e6c037 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:12.959624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a038b9-6366-4d25-99db-887fd48043a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:12.960263 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a038b9-6366-4d25-99db-887fd48043a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:12.960538 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a038b9-6366-4d25-99db-887fd48043a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:12.960807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a038b9-6366-4d25-99db-887fd48043a1 admin admin] PATH_INFO: `/v3/projects/b8f2ae169328402b8b7daf560548f3ac/users/d61464d586df46cdb7b14f47345c4d26/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:12.962982 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a038b9-6366-4d25-99db-887fd48043a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f2ae169328402b8b7daf560548f3ac, user_id=d61464d586df46cdb7b14f47345c4d26, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:12.965591 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a038b9-6366-4d25-99db-887fd48043a1 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:12.976189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1425/2865] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:12 2022] PUT /identity/v3/projects/b8f2ae169328402b8b7daf560548f3ac/users/d61464d586df46cdb7b14f47345c4d26/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:13.039458 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6b75d-9474-47ec-8cb0-e90df76b4abd None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:13.040049 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6b75d-9474-47ec-8cb0-e90df76b4abd None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:13.040049 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6b75d-9474-47ec-8cb0-e90df76b4abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:13.068170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-4fd6b75d-9474-47ec-8cb0-e90df76b4abd None None] MFA Rules not processed for user `d61464d586df46cdb7b14f47345c4d26`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:13.069400 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-4fd6b75d-9474-47ec-8cb0-e90df76b4abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:13.078463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:13.137195 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1441/2866] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:13 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:13.147929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a038b9-6366-4d25-99db-887fd48043a1 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:13.149611 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-b3a038b9-6366-4d25-99db-887fd48043a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:13.206474 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a038b9-6366-4d25-99db-887fd48043a1 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:13.210033 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53608f-17ae-48f1-a3c3-4b7854c3f914 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:13.210642 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53608f-17ae-48f1-a3c3-4b7854c3f914 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:13.210919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53608f-17ae-48f1-a3c3-4b7854c3f914 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:13.211187 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53608f-17ae-48f1-a3c3-4b7854c3f914 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:13.220494 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53608f-17ae-48f1-a3c3-4b7854c3f914 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:13.221956 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53608f-17ae-48f1-a3c3-4b7854c3f914 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:13.231689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1426/2867] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:13 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:13.312784 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6b75d-9474-47ec-8cb0-e90df76b4abd None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:13.322416 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6b75d-9474-47ec-8cb0-e90df76b4abd None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:13.326677 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4824cc0-7a10-469e-82b4-ba956f717177 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:13.327098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4824cc0-7a10-469e-82b4-ba956f717177 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:13.327404 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4824cc0-7a10-469e-82b4-ba956f717177 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:13.327659 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4824cc0-7a10-469e-82b4-ba956f717177 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:13.385804 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4824cc0-7a10-469e-82b4-ba956f717177 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:13.387270 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4824cc0-7a10-469e-82b4-ba956f717177 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:13.396572 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1442/2868] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:13 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:17.439355 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53608f-17ae-48f1-a3c3-4b7854c3f914 service cinder] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:17.496194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:17.509126 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53608f-17ae-48f1-a3c3-4b7854c3f914 service cinder] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:17.513640 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:17.514283 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:17.514562 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:17.514833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] PATH_INFO: `/v3/users/15d8231d652e450bb0255d05583e4bc9` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:17.515949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d8231d652e450bb0255d05583e4bc9)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:17.517885 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:17.563807 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d8231d652e450bb0255d05583e4bc9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:17.564483 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d8231d652e450bb0255d05583e4bc9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:17.576619 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d8231d652e450bb0255d05583e4bc9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:17.583941 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d8231d652e450bb0255d05583e4bc9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:17.584574 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d8231d652e450bb0255d05583e4bc9'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:17.597514 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1427/2869] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 09:04:17 2022] DELETE /identity/v3/users/15d8231d652e450bb0255d05583e4bc9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:17.602031 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d7c7c-5ff2-465c-8f63-393ef820c667 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:17.602559 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d7c7c-5ff2-465c-8f63-393ef820c667 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:17.602687 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d7c7c-5ff2-465c-8f63-393ef820c667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:17.621130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-2c6d7c7c-5ff2-465c-8f63-393ef820c667 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:17.622324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2c6d7c7c-5ff2-465c-8f63-393ef820c667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:17.681170 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1443/2870] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:17.690644 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:17.692571 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:17.748551 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14435eb8-62a3-4f5d-b775-4640e91fd488 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:17.752191 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb18115-5a61-4f84-aac2-d809b18d75d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:17.752787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb18115-5a61-4f84-aac2-d809b18d75d9 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:17.753069 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb18115-5a61-4f84-aac2-d809b18d75d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:17.753344 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb18115-5a61-4f84-aac2-d809b18d75d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:17.762702 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb18115-5a61-4f84-aac2-d809b18d75d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:17.764130 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb18115-5a61-4f84-aac2-d809b18d75d9 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:17.774201 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1428/2871] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:18.041503 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d7c7c-5ff2-465c-8f63-393ef820c667 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:18.043088 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-2c6d7c7c-5ff2-465c-8f63-393ef820c667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:18.056782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d7c7c-5ff2-465c-8f63-393ef820c667 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:18.061436 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:18.062098 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:18.062382 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:18.062647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] PATH_INFO: `/v3/projects/11d961e34acf45e29c6c24241ca9d438` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:18.063751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d961e34acf45e29c6c24241ca9d438)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:18.066558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:18.108972 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:18.116420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d961e34acf45e29c6c24241ca9d438', 'request_id': 'req-e922b00f-7f48-414e-be27-04f7f41ca842'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.122713 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d961e34acf45e29c6c24241ca9d438', 'request_id': 'req-e922b00f-7f48-414e-be27-04f7f41ca842'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.128262 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d961e34acf45e29c6c24241ca9d438', 'request_id': 'req-e922b00f-7f48-414e-be27-04f7f41ca842'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.140669 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] The token cache is being invalidate because project 11d961e34acf45e29c6c24241ca9d438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:04:18.140959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.141584 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.143106 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1444/2872] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 09:04:18 2022] DELETE /identity/v3/projects/11d961e34acf45e29c6c24241ca9d438 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:18.147684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb18115-5a61-4f84-aac2-d809b18d75d9 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:18.156977 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb18115-5a61-4f84-aac2-d809b18d75d9 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:18.211831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e0ea4-b5f3-40b5-8503-e7359854a08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:18.212445 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e0ea4-b5f3-40b5-8503-e7359854a08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:18.212719 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e0ea4-b5f3-40b5-8503-e7359854a08c admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:18.212990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e0ea4-b5f3-40b5-8503-e7359854a08c admin admin] PATH_INFO: `/v3/users/d61464d586df46cdb7b14f47345c4d26` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:18.214116 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e0ea4-b5f3-40b5-8503-e7359854a08c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d61464d586df46cdb7b14f47345c4d26)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:18.216080 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e0ea4-b5f3-40b5-8503-e7359854a08c admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:18.231014 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:18.261869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4f8e0ea4-b5f3-40b5-8503-e7359854a08c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd61464d586df46cdb7b14f47345c4d26'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.262522 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4f8e0ea4-b5f3-40b5-8503-e7359854a08c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd61464d586df46cdb7b14f47345c4d26'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.274769 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4f8e0ea4-b5f3-40b5-8503-e7359854a08c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd61464d586df46cdb7b14f47345c4d26'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.281598 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4f8e0ea4-b5f3-40b5-8503-e7359854a08c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd61464d586df46cdb7b14f47345c4d26'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.282266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-4f8e0ea4-b5f3-40b5-8503-e7359854a08c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd61464d586df46cdb7b14f47345c4d26'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.295266 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1429/2873] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 09:04:18 2022] DELETE /identity/v3/users/d61464d586df46cdb7b14f47345c4d26 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:18.561591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:18.563082 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:18.577184 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922b00f-7f48-414e-be27-04f7f41ca842 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:18.634395 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b71b9-d161-4429-ae57-dd2578928680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:18.635044 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b71b9-d161-4429-ae57-dd2578928680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:18.635333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b71b9-d161-4429-ae57-dd2578928680 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:18.635635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b71b9-d161-4429-ae57-dd2578928680 admin admin] PATH_INFO: `/v3/projects/b8f2ae169328402b8b7daf560548f3ac` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:18.636774 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b71b9-d161-4429-ae57-dd2578928680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f2ae169328402b8b7daf560548f3ac)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:18.638743 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b71b9-d161-4429-ae57-dd2578928680 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:18.685554 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-990b71b9-d161-4429-ae57-dd2578928680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f2ae169328402b8b7daf560548f3ac', 'request_id': 'req-990b71b9-d161-4429-ae57-dd2578928680'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.691600 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-990b71b9-d161-4429-ae57-dd2578928680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f2ae169328402b8b7daf560548f3ac', 'request_id': 'req-990b71b9-d161-4429-ae57-dd2578928680'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.697678 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-990b71b9-d161-4429-ae57-dd2578928680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f2ae169328402b8b7daf560548f3ac', 'request_id': 'req-990b71b9-d161-4429-ae57-dd2578928680'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.709937 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-990b71b9-d161-4429-ae57-dd2578928680 admin admin] The token cache is being invalidate because project b8f2ae169328402b8b7daf560548f3ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:04:18.710227 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-990b71b9-d161-4429-ae57-dd2578928680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.710850 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-990b71b9-d161-4429-ae57-dd2578928680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:18.712593 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1445/2874] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 09:04:18 2022] DELETE /identity/v3/projects/b8f2ae169328402b8b7daf560548f3ac => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:18.717529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db8b69-5d80-4124-a081-16f634f57aca None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:18.718046 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db8b69-5d80-4124-a081-16f634f57aca None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:18.718150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db8b69-5d80-4124-a081-16f634f57aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:18.737530 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-e5db8b69-5d80-4124-a081-16f634f57aca None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:18.739195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-e5db8b69-5d80-4124-a081-16f634f57aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:18.796202 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1430/2875] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:18.804221 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e53652-9687-49f5-8352-2e4ef70f2174 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:18.804509 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e53652-9687-49f5-8352-2e4ef70f2174 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:18.804782 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e53652-9687-49f5-8352-2e4ef70f2174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:18.824527 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-18e53652-9687-49f5-8352-2e4ef70f2174 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:18.825753 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-18e53652-9687-49f5-8352-2e4ef70f2174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:18.833595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1446/2876] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:18.837798 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db8b69-5d80-4124-a081-16f634f57aca None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:18.847184 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db8b69-5d80-4124-a081-16f634f57aca None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:18.851720 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd2018-a0a1-4255-a0f1-4f3a8163b3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:18.852309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd2018-a0a1-4255-a0f1-4f3a8163b3cb admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:18.852589 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd2018-a0a1-4255-a0f1-4f3a8163b3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:18.852857 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd2018-a0a1-4255-a0f1-4f3a8163b3cb admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:18.853488 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd2018-a0a1-4255-a0f1-4f3a8163b3cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:04:18.853826 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd2018-a0a1-4255-a0f1-4f3a8163b3cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:18.855402 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd2018-a0a1-4255-a0f1-4f3a8163b3cb admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:18.867097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1431/2877] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 09:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:04:18.871185 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e53652-9687-49f5-8352-2e4ef70f2174 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:18.880979 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e53652-9687-49f5-8352-2e4ef70f2174 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:18.885634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510b137-55ec-4e61-879b-77a9de557df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:18.886246 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510b137-55ec-4e61-879b-77a9de557df7 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:18.886528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510b137-55ec-4e61-879b-77a9de557df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:18.886802 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510b137-55ec-4e61-879b-77a9de557df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:18.887429 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510b137-55ec-4e61-879b-77a9de557df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:18.889354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510b137-55ec-4e61-879b-77a9de557df7 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:18.916080 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1447/2878] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 09:04:18 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:04:18.920083 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd2018-a0a1-4255-a0f1-4f3a8163b3cb admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:18.930537 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd2018-a0a1-4255-a0f1-4f3a8163b3cb admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:18.985104 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfe1fc-6dde-499b-8a04-ba1960486318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:18.985708 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfe1fc-6dde-499b-8a04-ba1960486318 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:18.985996 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfe1fc-6dde-499b-8a04-ba1960486318 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:18.986268 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfe1fc-6dde-499b-8a04-ba1960486318 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:18.986880 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abfe1fc-6dde-499b-8a04-ba1960486318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:18.989456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abfe1fc-6dde-499b-8a04-ba1960486318 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:19.010000 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1432/2879] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 09:04:18 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:04:19.014346 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510b137-55ec-4e61-879b-77a9de557df7 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:19.024439 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510b137-55ec-4e61-879b-77a9de557df7 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:19.029109 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c652d5-0f84-48b5-ae58-63b1c1bec382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:19.029722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c652d5-0f84-48b5-ae58-63b1c1bec382 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:19.030021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c652d5-0f84-48b5-ae58-63b1c1bec382 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:19.030297 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c652d5-0f84-48b5-ae58-63b1c1bec382 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:19.030938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c652d5-0f84-48b5-ae58-63b1c1bec382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:19.032573 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c652d5-0f84-48b5-ae58-63b1c1bec382 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:19.042960 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1448/2880] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:19.046811 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfe1fc-6dde-499b-8a04-ba1960486318 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:19.057278 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfe1fc-6dde-499b-8a04-ba1960486318 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:19.061904 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936c60e-3d63-4ef1-967f-1e20e3e9bfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:19.062497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936c60e-3d63-4ef1-967f-1e20e3e9bfa8 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:19.062786 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936c60e-3d63-4ef1-967f-1e20e3e9bfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:19.063060 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936c60e-3d63-4ef1-967f-1e20e3e9bfa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:19.063788 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936c60e-3d63-4ef1-967f-1e20e3e9bfa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:19.065399 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936c60e-3d63-4ef1-967f-1e20e3e9bfa8 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:19.070762 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:19.077279 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1433/2881] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:19.081482 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c652d5-0f84-48b5-ae58-63b1c1bec382 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:19.091627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c652d5-0f84-48b5-ae58-63b1c1bec382 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:19.096248 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7d421-41d5-41fb-b4f6-cbe6b1516493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:19.096886 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7d421-41d5-41fb-b4f6-cbe6b1516493 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:19.097167 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7d421-41d5-41fb-b4f6-cbe6b1516493 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:19.097440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7d421-41d5-41fb-b4f6-cbe6b1516493 admin admin] PATH_INFO: `/v3/projects/9a16532337f04fe5b61f3109586baaa2/users/ba83178f22b14bdb83c83aa8b055326a/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:19.112157 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7d421-41d5-41fb-b4f6-cbe6b1516493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a16532337f04fe5b61f3109586baaa2, user_id=ba83178f22b14bdb83c83aa8b055326a, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:19.115433 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7d421-41d5-41fb-b4f6-cbe6b1516493 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:19.118437 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:19.130250 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1449/2882] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:19 2022] PUT /identity/v3/projects/9a16532337f04fe5b61f3109586baaa2/users/ba83178f22b14bdb83c83aa8b055326a/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:19.141907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5c38a-5014-4004-97bf-d6e5f0013d9d None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:19.142198 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5c38a-5014-4004-97bf-d6e5f0013d9d None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:19.142473 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5c38a-5014-4004-97bf-d6e5f0013d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:19.170744 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-38d5c38a-5014-4004-97bf-d6e5f0013d9d None None] MFA Rules not processed for user `ba83178f22b14bdb83c83aa8b055326a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:19.171966 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-38d5c38a-5014-4004-97bf-d6e5f0013d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:19.234318 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1434/2883] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:19.239432 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7d421-41d5-41fb-b4f6-cbe6b1516493 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:19.303357 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7d421-41d5-41fb-b4f6-cbe6b1516493 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:19.308181 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d61f2-4fe5-4e26-96e0-8581b7918920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:19.308790 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d61f2-4fe5-4e26-96e0-8581b7918920 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:19.309074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d61f2-4fe5-4e26-96e0-8581b7918920 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:19.309352 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d61f2-4fe5-4e26-96e0-8581b7918920 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:19.310037 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d61f2-4fe5-4e26-96e0-8581b7918920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:19.311957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d61f2-4fe5-4e26-96e0-8581b7918920 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:19.338193 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1450/2884] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 09:04:19 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:04:19.342204 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5c38a-5014-4004-97bf-d6e5f0013d9d None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:19.353209 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5c38a-5014-4004-97bf-d6e5f0013d9d None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:19.406955 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97415153-cd01-4394-aa47-4e77b919b69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:19.407548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97415153-cd01-4394-aa47-4e77b919b69f admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:19.407827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97415153-cd01-4394-aa47-4e77b919b69f admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:19.408098 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97415153-cd01-4394-aa47-4e77b919b69f admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:19.408711 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97415153-cd01-4394-aa47-4e77b919b69f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:19.411197 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97415153-cd01-4394-aa47-4e77b919b69f admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:19.434924 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1435/2885] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 09:04:19 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:04:19.436223 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d61f2-4fe5-4e26-96e0-8581b7918920 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:19.446674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d61f2-4fe5-4e26-96e0-8581b7918920 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:19.451311 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1588a-e2ab-411b-87ac-3035ef7a5ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:19.451908 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1588a-e2ab-411b-87ac-3035ef7a5ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:19.452191 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1588a-e2ab-411b-87ac-3035ef7a5ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:19.452468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1588a-e2ab-411b-87ac-3035ef7a5ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:19.453113 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1588a-e2ab-411b-87ac-3035ef7a5ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:19.454755 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1588a-e2ab-411b-87ac-3035ef7a5ae3 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:19.466116 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1451/2886] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:19.470066 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97415153-cd01-4394-aa47-4e77b919b69f admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:19.480560 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97415153-cd01-4394-aa47-4e77b919b69f admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:19.485203 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b435ad-e275-4dbb-a684-a3fe813c8994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:19.485891 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b435ad-e275-4dbb-a684-a3fe813c8994 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:19.486176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b435ad-e275-4dbb-a684-a3fe813c8994 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:19.486452 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b435ad-e275-4dbb-a684-a3fe813c8994 admin admin] PATH_INFO: `/v3/projects/ca15b9b94032471cbce0bca7067fdaf6/users/b3a39b7d654143e0a828fd324c02d15f/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:19.503354 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b435ad-e275-4dbb-a684-a3fe813c8994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca15b9b94032471cbce0bca7067fdaf6, user_id=b3a39b7d654143e0a828fd324c02d15f, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:19.506396 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b435ad-e275-4dbb-a684-a3fe813c8994 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:19.518949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1436/2887] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:19 2022] PUT /identity/v3/projects/ca15b9b94032471cbce0bca7067fdaf6/users/b3a39b7d654143e0a828fd324c02d15f/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:19.523394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1588a-e2ab-411b-87ac-3035ef7a5ae3 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:19.533197 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1588a-e2ab-411b-87ac-3035ef7a5ae3 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:19.589448 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a66c25-b5d7-4084-904c-eff48d0dec83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:19.590003 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a66c25-b5d7-4084-904c-eff48d0dec83 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:19.590283 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a66c25-b5d7-4084-904c-eff48d0dec83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:19.590555 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a66c25-b5d7-4084-904c-eff48d0dec83 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:19.591182 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a66c25-b5d7-4084-904c-eff48d0dec83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:19.592765 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a66c25-b5d7-4084-904c-eff48d0dec83 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:19.602635 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1452/2888] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:19.606746 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b435ad-e275-4dbb-a684-a3fe813c8994 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:19.616852 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b435ad-e275-4dbb-a684-a3fe813c8994 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:19.621529 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cee586-4f8b-4f33-99f1-ecbe80e44e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:19.622208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cee586-4f8b-4f33-99f1-ecbe80e44e79 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:19.622488 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cee586-4f8b-4f33-99f1-ecbe80e44e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:19.622758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cee586-4f8b-4f33-99f1-ecbe80e44e79 admin admin] PATH_INFO: `/v3/projects/ca15b9b94032471cbce0bca7067fdaf6/users/b3a39b7d654143e0a828fd324c02d15f/roles/f763a168cafd44ffb29843b3716d5f50` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:19.624649 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cee586-4f8b-4f33-99f1-ecbe80e44e79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca15b9b94032471cbce0bca7067fdaf6, user_id=b3a39b7d654143e0a828fd324c02d15f, role_id=f763a168cafd44ffb29843b3716d5f50)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:19.627304 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cee586-4f8b-4f33-99f1-ecbe80e44e79 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:19.637760 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1437/2889] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:19 2022] PUT /identity/v3/projects/ca15b9b94032471cbce0bca7067fdaf6/users/b3a39b7d654143e0a828fd324c02d15f/roles/f763a168cafd44ffb29843b3716d5f50 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:19.648312 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e796a-734f-4414-8ab1-5ffbabc48b24 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:19.648797 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e796a-734f-4414-8ab1-5ffbabc48b24 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:19.648912 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e796a-734f-4414-8ab1-5ffbabc48b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:19.677637 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-cf9e796a-734f-4414-8ab1-5ffbabc48b24 None None] MFA Rules not processed for user `b3a39b7d654143e0a828fd324c02d15f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:19.678863 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-cf9e796a-734f-4414-8ab1-5ffbabc48b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:19.728463 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:19.746547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1453/2890] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:19.757787 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cee586-4f8b-4f33-99f1-ecbe80e44e79 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:19.759593 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-88cee586-4f8b-4f33-99f1-ecbe80e44e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:19.805958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:19.818606 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cee586-4f8b-4f33-99f1-ecbe80e44e79 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:19.822280 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e515c-087e-4661-85a5-6da1c88939ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:19.822876 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e515c-087e-4661-85a5-6da1c88939ff service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:19.823156 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e515c-087e-4661-85a5-6da1c88939ff service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:19.823427 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e515c-087e-4661-85a5-6da1c88939ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:19.877844 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e515c-087e-4661-85a5-6da1c88939ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:19.879703 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e515c-087e-4661-85a5-6da1c88939ff service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:19.888378 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1438/2891] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:22.583072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e796a-734f-4414-8ab1-5ffbabc48b24 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:22.643547 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e796a-734f-4414-8ab1-5ffbabc48b24 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:22.648637 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:22.649258 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:22.649535 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:22.649839 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] PATH_INFO: `/v3/users/ba83178f22b14bdb83c83aa8b055326a` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:22.650956 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba83178f22b14bdb83c83aa8b055326a)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:22.652879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:22.697391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba83178f22b14bdb83c83aa8b055326a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:22.698095 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba83178f22b14bdb83c83aa8b055326a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:22.705099 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba83178f22b14bdb83c83aa8b055326a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:22.705826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba83178f22b14bdb83c83aa8b055326a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:22.719568 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba83178f22b14bdb83c83aa8b055326a'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:22.732011 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1454/2892] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 09:04:22 2022] DELETE /identity/v3/users/ba83178f22b14bdb83c83aa8b055326a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:22.736549 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a13d1e-8063-4385-ad19-beda3195d29b None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:22.737016 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a13d1e-8063-4385-ad19-beda3195d29b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:22.737150 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a13d1e-8063-4385-ad19-beda3195d29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:22.755907 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-f0a13d1e-8063-4385-ad19-beda3195d29b None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:22.757097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f0a13d1e-8063-4385-ad19-beda3195d29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:22.813479 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1439/2893] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:22.823501 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:22.824906 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:22.880172 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655058c-288e-4a93-98da-95547c69d52e admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:22.883766 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29648a59-d895-408c-babe-7ff5acba85f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:22.884358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29648a59-d895-408c-babe-7ff5acba85f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:22.884634 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29648a59-d895-408c-babe-7ff5acba85f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:22.884904 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29648a59-d895-408c-babe-7ff5acba85f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:22.893528 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29648a59-d895-408c-babe-7ff5acba85f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:22.895627 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29648a59-d895-408c-babe-7ff5acba85f7 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:22.904775 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1455/2894] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:23.171594 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a13d1e-8063-4385-ad19-beda3195d29b None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:23.173139 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-f0a13d1e-8063-4385-ad19-beda3195d29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:23.186665 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a13d1e-8063-4385-ad19-beda3195d29b None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:23.192597 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:23.193444 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:23.193749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:23.194070 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] PATH_INFO: `/v3/projects/9a16532337f04fe5b61f3109586baaa2` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:23.195132 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a16532337f04fe5b61f3109586baaa2)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:23.197021 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:23.243689 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a16532337f04fe5b61f3109586baaa2', 'request_id': 'req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.249764 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a16532337f04fe5b61f3109586baaa2', 'request_id': 'req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.256065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a16532337f04fe5b61f3109586baaa2', 'request_id': 'req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.267757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] The token cache is being invalidate because project 9a16532337f04fe5b61f3109586baaa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:04:23.268049 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.268673 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.270196 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1440/2895] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 09:04:23 2022] DELETE /identity/v3/projects/9a16532337f04fe5b61f3109586baaa2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:23.274210 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29648a59-d895-408c-babe-7ff5acba85f7 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:23.283730 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29648a59-d895-408c-babe-7ff5acba85f7 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:23.292386 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:23.342595 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa7995-4cbc-4190-8d36-370266666782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:23.343255 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7995-4cbc-4190-8d36-370266666782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:23.343538 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7995-4cbc-4190-8d36-370266666782 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:23.343889 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7995-4cbc-4190-8d36-370266666782 admin admin] PATH_INFO: `/v3/users/b3a39b7d654143e0a828fd324c02d15f` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:23.344975 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa7995-4cbc-4190-8d36-370266666782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a39b7d654143e0a828fd324c02d15f)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:23.346951 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa7995-4cbc-4190-8d36-370266666782 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:23.393079 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a7fa7995-4cbc-4190-8d36-370266666782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a39b7d654143e0a828fd324c02d15f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.393747 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a7fa7995-4cbc-4190-8d36-370266666782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a39b7d654143e0a828fd324c02d15f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.400710 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a7fa7995-4cbc-4190-8d36-370266666782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a39b7d654143e0a828fd324c02d15f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.401345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a7fa7995-4cbc-4190-8d36-370266666782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a39b7d654143e0a828fd324c02d15f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.413958 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-a7fa7995-4cbc-4190-8d36-370266666782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a39b7d654143e0a828fd324c02d15f'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.427533 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1456/2896] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 09:04:23 2022] DELETE /identity/v3/users/b3a39b7d654143e0a828fd324c02d15f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:23.691733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:23.693194 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:23.707831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9caf4b5-f707-4f27-9ccd-afef13caa2f2 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:23.725245 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:23.765779 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:23.766429 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:23.766669 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:23.766939 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2 admin admin] PATH_INFO: `/v3/projects/ca15b9b94032471cbce0bca7067fdaf6` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:23.768030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca15b9b94032471cbce0bca7067fdaf6)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:23.770454 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:23.814933 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca15b9b94032471cbce0bca7067fdaf6', 'request_id': 'req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.820944 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca15b9b94032471cbce0bca7067fdaf6', 'request_id': 'req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.826497 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca15b9b94032471cbce0bca7067fdaf6', 'request_id': 'req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.839467 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2 admin admin] The token cache is being invalidate because project ca15b9b94032471cbce0bca7067fdaf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87946) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:04:23.839758 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.840377 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-811ddafd-9fd0-42ff-ad34-a57a2ccfa1b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:23.841856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1441/2897] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 09:04:23 2022] DELETE /identity/v3/projects/ca15b9b94032471cbce0bca7067fdaf6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:23.846778 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31e952-819a-4b0c-a68b-7bbcf99fe572 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:23.847253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31e952-819a-4b0c-a68b-7bbcf99fe572 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:23.847391 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31e952-819a-4b0c-a68b-7bbcf99fe572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:23.867027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-0e31e952-819a-4b0c-a68b-7bbcf99fe572 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:23.868175 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-0e31e952-819a-4b0c-a68b-7bbcf99fe572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:23.924534 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1457/2898] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:23.932286 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931ab9-65d8-43d7-84a6-9db92d6c7269 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:23.932577 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931ab9-65d8-43d7-84a6-9db92d6c7269 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:23.932858 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae931ab9-65d8-43d7-84a6-9db92d6c7269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:23.952276 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-ae931ab9-65d8-43d7-84a6-9db92d6c7269 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:23.953484 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-ae931ab9-65d8-43d7-84a6-9db92d6c7269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:23.962112 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1442/2899] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:23.966232 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31e952-819a-4b0c-a68b-7bbcf99fe572 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:23.976393 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31e952-819a-4b0c-a68b-7bbcf99fe572 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:23.981159 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2914cb7-acbf-4598-ad36-9f833347bd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:23.981772 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2914cb7-acbf-4598-ad36-9f833347bd24 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:23.982052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2914cb7-acbf-4598-ad36-9f833347bd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:23.982320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2914cb7-acbf-4598-ad36-9f833347bd24 admin admin] PATH_INFO: `/v3/domains` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:23.982954 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2914cb7-acbf-4598-ad36-9f833347bd24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87945) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:04:23.983252 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2914cb7-acbf-4598-ad36-9f833347bd24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:23.984845 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2914cb7-acbf-4598-ad36-9f833347bd24 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:23.996070 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1458/2900] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 09:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:04:24.000295 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae931ab9-65d8-43d7-84a6-9db92d6c7269 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:24.009382 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae931ab9-65d8-43d7-84a6-9db92d6c7269 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:24.013990 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ec844-132c-47ef-a9f7-2362177043cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:24.014581 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ec844-132c-47ef-a9f7-2362177043cf admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:24.014860 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ec844-132c-47ef-a9f7-2362177043cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:24.015133 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ec844-132c-47ef-a9f7-2362177043cf admin admin] PATH_INFO: `/v3/projects` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:24.015757 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ec844-132c-47ef-a9f7-2362177043cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:24.017641 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ec844-132c-47ef-a9f7-2362177043cf admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:24.044728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1443/2901] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 09:04:23 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:04:24.048751 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2914cb7-acbf-4598-ad36-9f833347bd24 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:24.058553 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2914cb7-acbf-4598-ad36-9f833347bd24 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:24.074832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:24.116870 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7b35-d92d-4482-943f-c2b4e161f895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:24.117523 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7b35-d92d-4482-943f-c2b4e161f895 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:24.117830 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7b35-d92d-4482-943f-c2b4e161f895 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:24.118111 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f7b35-d92d-4482-943f-c2b4e161f895 admin admin] PATH_INFO: `/v3/users` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:24.118750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f7b35-d92d-4482-943f-c2b4e161f895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:24.120688 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f7b35-d92d-4482-943f-c2b4e161f895 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:24.142042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1459/2902] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 09:04:24 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:04:24.146313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ec844-132c-47ef-a9f7-2362177043cf admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:24.156464 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ec844-132c-47ef-a9f7-2362177043cf admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:24.161548 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c712fcf-f732-4316-b760-564de5278cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:24.162174 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c712fcf-f732-4316-b760-564de5278cbb admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:24.162457 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c712fcf-f732-4316-b760-564de5278cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:24.162730 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c712fcf-f732-4316-b760-564de5278cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:24.163390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c712fcf-f732-4316-b760-564de5278cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:24.164972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c712fcf-f732-4316-b760-564de5278cbb admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:24.174810 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1444/2903] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:24.179560 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7b35-d92d-4482-943f-c2b4e161f895 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:24.189513 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f7b35-d92d-4482-943f-c2b4e161f895 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:24.194168 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b078c-ad49-4a5a-ab06-cb65c81567a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:24.194759 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b078c-ad49-4a5a-ab06-cb65c81567a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:24.195041 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b078c-ad49-4a5a-ab06-cb65c81567a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:24.195317 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b078c-ad49-4a5a-ab06-cb65c81567a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:24.195959 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b078c-ad49-4a5a-ab06-cb65c81567a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:24.197563 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b078c-ad49-4a5a-ab06-cb65c81567a9 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:24.210213 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1460/2904] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:24.212388 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c712fcf-f732-4316-b760-564de5278cbb admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:24.222265 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c712fcf-f732-4316-b760-564de5278cbb admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:24.227624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0203-3bfd-439c-af26-3d6277696db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:24.228269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0203-3bfd-439c-af26-3d6277696db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:24.228552 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0203-3bfd-439c-af26-3d6277696db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:24.228832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0203-3bfd-439c-af26-3d6277696db5 admin admin] PATH_INFO: `/v3/projects/37c76a650ce1442ca18b129c109d1dbe/users/6669ac4df7fb4a89acd662b2f8b5c07d/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:24.244012 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c0203-3bfd-439c-af26-3d6277696db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c76a650ce1442ca18b129c109d1dbe, user_id=6669ac4df7fb4a89acd662b2f8b5c07d, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:24.246734 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c0203-3bfd-439c-af26-3d6277696db5 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:24.259401 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1445/2905] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:24 2022] PUT /identity/v3/projects/37c76a650ce1442ca18b129c109d1dbe/users/6669ac4df7fb4a89acd662b2f8b5c07d/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:24.270515 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada5061-fde1-4988-8b42-4669a7012946 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:24.270808 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada5061-fde1-4988-8b42-4669a7012946 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:24.271090 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada5061-fde1-4988-8b42-4669a7012946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:24.299027 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-fada5061-fde1-4988-8b42-4669a7012946 None None] MFA Rules not processed for user `6669ac4df7fb4a89acd662b2f8b5c07d`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:24.300243 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fada5061-fde1-4988-8b42-4669a7012946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:24.364837 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1461/2906] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:24.375390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0203-3bfd-439c-af26-3d6277696db5 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:24.377065 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2f1c0203-3bfd-439c-af26-3d6277696db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:24.428997 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:24.435176 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0203-3bfd-439c-af26-3d6277696db5 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:24.439519 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34bcd2-0917-4b7e-83a7-15f9f67ae1c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:24.440113 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34bcd2-0917-4b7e-83a7-15f9f67ae1c4 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:24.440390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34bcd2-0917-4b7e-83a7-15f9f67ae1c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:24.440791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34bcd2-0917-4b7e-83a7-15f9f67ae1c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:24.448916 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34bcd2-0917-4b7e-83a7-15f9f67ae1c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:24.450368 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34bcd2-0917-4b7e-83a7-15f9f67ae1c4 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:24.458965 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1446/2907] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:25.806983 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3749563-1337-4c96-923a-6873e1877370 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:25.807495 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3749563-1337-4c96-923a-6873e1877370 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:25.807610 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3749563-1337-4c96-923a-6873e1877370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:25.825893 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-c3749563-1337-4c96-923a-6873e1877370 None None] MFA Rules not processed for user `6669ac4df7fb4a89acd662b2f8b5c07d`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:25.827075 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c3749563-1337-4c96-923a-6873e1877370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:25.833652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1462/2908] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:34.455038 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=5 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:35.821229 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=60,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:41.788261 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed1ff9-55a1-4183-a3aa-fc410148ee10 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:41.788756 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed1ff9-55a1-4183-a3aa-fc410148ee10 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:41.788866 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed1ff9-55a1-4183-a3aa-fc410148ee10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:41.837824 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-4bed1ff9-55a1-4183-a3aa-fc410148ee10 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:41.839018 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-4bed1ff9-55a1-4183-a3aa-fc410148ee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:41.901395 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1447/2909] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 09:04:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:41.946938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3749563-1337-4c96-923a-6873e1877370 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:41.948324 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-c3749563-1337-4c96-923a-6873e1877370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:42.037879 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3749563-1337-4c96-923a-6873e1877370 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:42.042880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def2765-b297-45f0-9bb5-eea4ebf5e2d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:42.043484 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def2765-b297-45f0-9bb5-eea4ebf5e2d4 service swift] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:42.043767 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def2765-b297-45f0-9bb5-eea4ebf5e2d4 service swift] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:42.044043 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def2765-b297-45f0-9bb5-eea4ebf5e2d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:42.052077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def2765-b297-45f0-9bb5-eea4ebf5e2d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:42.053652 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def2765-b297-45f0-9bb5-eea4ebf5e2d4 service swift] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:42.061868 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1463/2910] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 09:04:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 09:04:42.564309 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaca98c-ba78-4ad3-8ad3-3926e63f2425 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:42.564613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaca98c-ba78-4ad3-8ad3-3926e63f2425 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:42.564909 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaca98c-ba78-4ad3-8ad3-3926e63f2425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:42.584048 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-7eaca98c-ba78-4ad3-8ad3-3926e63f2425 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:42.585222 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-7eaca98c-ba78-4ad3-8ad3-3926e63f2425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:42.592414 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1448/2911] 10.222.0.26 () {58 vars in 967 bytes} [Tue Oct 4 09:04:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:04:42.604259 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def2765-b297-45f0-9bb5-eea4ebf5e2d4 service swift] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:42.615441 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def2765-b297-45f0-9bb5-eea4ebf5e2d4 service swift] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:42.620543 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d36b13-2f9e-4586-ae8d-bccab32f2b67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:42.621139 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d36b13-2f9e-4586-ae8d-bccab32f2b67 service swift] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:42.621445 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d36b13-2f9e-4586-ae8d-bccab32f2b67 service swift] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:42.621784 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d36b13-2f9e-4586-ae8d-bccab32f2b67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:42.630736 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d36b13-2f9e-4586-ae8d-bccab32f2b67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:42.632465 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d36b13-2f9e-4586-ae8d-bccab32f2b67 service swift] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:42.640331 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1464/2912] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 09:04:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 09:04:42.862244 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68e0f4-7583-4314-b023-0ae30fff3669 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:42.862761 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68e0f4-7583-4314-b023-0ae30fff3669 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:42.862863 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68e0f4-7583-4314-b023-0ae30fff3669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:42.881959 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-bf68e0f4-7583-4314-b023-0ae30fff3669 None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:42.883134 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-bf68e0f4-7583-4314-b023-0ae30fff3669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:42.890060 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1449/2913] 10.222.0.26 () {58 vars in 1009 bytes} [Tue Oct 4 09:04:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:42.901756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d36b13-2f9e-4586-ae8d-bccab32f2b67 service swift] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:42.912722 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d36b13-2f9e-4586-ae8d-bccab32f2b67 service swift] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:42.917755 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221121b1-98b3-4347-800f-bf570d92f651 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:42.918345 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221121b1-98b3-4347-800f-bf570d92f651 service swift] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:42.918623 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221121b1-98b3-4347-800f-bf570d92f651 service swift] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:42.918903 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221121b1-98b3-4347-800f-bf570d92f651 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:42.927462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221121b1-98b3-4347-800f-bf570d92f651 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:42.928974 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221121b1-98b3-4347-800f-bf570d92f651 service swift] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:42.936826 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1465/2914] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 09:04:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 09:04:43.247419 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aa222-bc2d-470d-abb4-1a802fa9e53b None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:43.247706 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aa222-bc2d-470d-abb4-1a802fa9e53b None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:43.247975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aa222-bc2d-470d-abb4-1a802fa9e53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:43.266692 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-888aa222-bc2d-470d-abb4-1a802fa9e53b None None] MFA Rules not processed for user `595dac02b846405cab34dcf898ce5b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:43.267874 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-888aa222-bc2d-470d-abb4-1a802fa9e53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:43.275331 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1450/2915] 10.222.0.26 () {58 vars in 967 bytes} [Tue Oct 4 09:04:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:43.287911 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221121b1-98b3-4347-800f-bf570d92f651 service swift] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:43.298371 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221121b1-98b3-4347-800f-bf570d92f651 service swift] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:43.303474 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c1ddc-812b-4178-a6fb-fcfc255fde5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b232b71c77a24be5bfcdb6aeb5807f90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:43.304064 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c1ddc-812b-4178-a6fb-fcfc255fde5e service swift] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:43.304344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c1ddc-812b-4178-a6fb-fcfc255fde5e service swift] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:43.304617 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c1ddc-812b-4178-a6fb-fcfc255fde5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:43.312686 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c1ddc-812b-4178-a6fb-fcfc255fde5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:43.315358 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c1ddc-812b-4178-a6fb-fcfc255fde5e service swift] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:43.323926 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1466/2916] 10.222.0.26 () {60 vars in 1384 bytes} [Tue Oct 4 09:04:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 09:04:45.398846 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aa222-bc2d-470d-abb4-1a802fa9e53b None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:45.460978 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aa222-bc2d-470d-abb4-1a802fa9e53b None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:45.465818 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:45.466446 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:45.466726 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:45.466997 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] PATH_INFO: `/v3/users/6669ac4df7fb4a89acd662b2f8b5c07d` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:45.468144 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6669ac4df7fb4a89acd662b2f8b5c07d)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:45.470100 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:45.515075 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6669ac4df7fb4a89acd662b2f8b5c07d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:45.515724 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6669ac4df7fb4a89acd662b2f8b5c07d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:45.521026 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:45.528912 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6669ac4df7fb4a89acd662b2f8b5c07d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:45.535670 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6669ac4df7fb4a89acd662b2f8b5c07d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:45.536321 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6669ac4df7fb4a89acd662b2f8b5c07d'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:45.549240 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1451/2917] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 09:04:45 2022] DELETE /identity/v3/users/6669ac4df7fb4a89acd662b2f8b5c07d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:45.554207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c0162-4a6b-44f1-9476-481aff25960d None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:45.554714 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c0162-4a6b-44f1-9476-481aff25960d None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:45.554832 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c0162-4a6b-44f1-9476-481aff25960d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:45.574155 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-fb6c0162-4a6b-44f1-9476-481aff25960d None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:45.575354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fb6c0162-4a6b-44f1-9476-481aff25960d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:45.635969 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1467/2918] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:45.646927 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:45.648324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:45.725410 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0d8d2-9438-42ff-bb5d-71551969a10a admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:45.729023 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b8f63-e045-49e7-8c2e-8d92381dbabe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:45.729624 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b8f63-e045-49e7-8c2e-8d92381dbabe service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:45.729929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b8f63-e045-49e7-8c2e-8d92381dbabe service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:45.730200 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b8f63-e045-49e7-8c2e-8d92381dbabe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:45.738868 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b8f63-e045-49e7-8c2e-8d92381dbabe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:45.740813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b8f63-e045-49e7-8c2e-8d92381dbabe service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:45.749919 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1452/2919] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:45.913939 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c0162-4a6b-44f1-9476-481aff25960d None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:45.915378 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-fb6c0162-4a6b-44f1-9476-481aff25960d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:45.927247 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c0162-4a6b-44f1-9476-481aff25960d None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:45.932425 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a2c60-b45e-43fe-8839-006e555ef390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:45.933052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a2c60-b45e-43fe-8839-006e555ef390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:45.933344 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a2c60-b45e-43fe-8839-006e555ef390 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:45.933599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a2c60-b45e-43fe-8839-006e555ef390 admin admin] PATH_INFO: `/v3/projects/37c76a650ce1442ca18b129c109d1dbe` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:45.934741 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a2c60-b45e-43fe-8839-006e555ef390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c76a650ce1442ca18b129c109d1dbe)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:45.936656 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a2c60-b45e-43fe-8839-006e555ef390 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:45.967207 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:45.984841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e35a2c60-b45e-43fe-8839-006e555ef390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c76a650ce1442ca18b129c109d1dbe', 'request_id': 'req-e35a2c60-b45e-43fe-8839-006e555ef390'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:45.991494 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e35a2c60-b45e-43fe-8839-006e555ef390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c76a650ce1442ca18b129c109d1dbe', 'request_id': 'req-e35a2c60-b45e-43fe-8839-006e555ef390'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:45.997039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e35a2c60-b45e-43fe-8839-006e555ef390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c76a650ce1442ca18b129c109d1dbe', 'request_id': 'req-e35a2c60-b45e-43fe-8839-006e555ef390'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:46.008759 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e35a2c60-b45e-43fe-8839-006e555ef390 admin admin] The token cache is being invalidate because project 37c76a650ce1442ca18b129c109d1dbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:04:46.009077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e35a2c60-b45e-43fe-8839-006e555ef390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:46.009725 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-e35a2c60-b45e-43fe-8839-006e555ef390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:04:46.011208 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1468/2920] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 09:04:45 2022] DELETE /identity/v3/projects/37c76a650ce1442ca18b129c109d1dbe => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:04:46.016060 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda392-1fae-4e94-81fb-59b364447213 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:46.016351 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda392-1fae-4e94-81fb-59b364447213 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:46.016631 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda392-1fae-4e94-81fb-59b364447213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:46.035729 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-64dda392-1fae-4e94-81fb-59b364447213 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:46.037432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-64dda392-1fae-4e94-81fb-59b364447213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:46.094247 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1453/2921] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:46.103304 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c914b-9aae-4e07-82a8-30916b04fca8 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:46.103591 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c914b-9aae-4e07-82a8-30916b04fca8 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:46.103869 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c914b-9aae-4e07-82a8-30916b04fca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:46.123922 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-ee0c914b-9aae-4e07-82a8-30916b04fca8 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:46.125130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ee0c914b-9aae-4e07-82a8-30916b04fca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:46.132962 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1469/2922] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 09:04:46.137237 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dda392-1fae-4e94-81fb-59b364447213 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:46.146812 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dda392-1fae-4e94-81fb-59b364447213 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:46.151339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dadeda-fc43-4df0-b63a-393ead52e95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:46.151929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dadeda-fc43-4df0-b63a-393ead52e95e admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:46.152208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dadeda-fc43-4df0-b63a-393ead52e95e admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:46.152481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dadeda-fc43-4df0-b63a-393ead52e95e admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:46.153117 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dadeda-fc43-4df0-b63a-393ead52e95e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:04:46.153422 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dadeda-fc43-4df0-b63a-393ead52e95e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:46.155071 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dadeda-fc43-4df0-b63a-393ead52e95e admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:46.166937 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1454/2923] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 09:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:04:46.170950 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c914b-9aae-4e07-82a8-30916b04fca8 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:46.180614 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c914b-9aae-4e07-82a8-30916b04fca8 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:46.185235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88307d-88c0-43c2-8869-070bff67e4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:46.185860 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88307d-88c0-43c2-8869-070bff67e4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:46.186144 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88307d-88c0-43c2-8869-070bff67e4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:46.186414 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88307d-88c0-43c2-8869-070bff67e4a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:46.187039 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88307d-88c0-43c2-8869-070bff67e4a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:46.188933 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88307d-88c0-43c2-8869-070bff67e4a0 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:46.215966 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1470/2924] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 09:04:46 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:04:46.219958 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dadeda-fc43-4df0-b63a-393ead52e95e admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:46.230241 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dadeda-fc43-4df0-b63a-393ead52e95e admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:46.268789 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:46.287815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586172a-04e9-4466-ba5d-3285f3bb9ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:46.288413 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586172a-04e9-4466-ba5d-3285f3bb9ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:46.288695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586172a-04e9-4466-ba5d-3285f3bb9ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:46.288972 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586172a-04e9-4466-ba5d-3285f3bb9ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:46.289604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586172a-04e9-4466-ba5d-3285f3bb9ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:46.291540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586172a-04e9-4466-ba5d-3285f3bb9ea9 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:46.311747 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1455/2925] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 09:04:46 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:04:46.315881 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88307d-88c0-43c2-8869-070bff67e4a0 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:46.326187 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88307d-88c0-43c2-8869-070bff67e4a0 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:46.330756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4ecc6-f3da-472c-9b9e-e74fc404fbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:46.331390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4ecc6-f3da-472c-9b9e-e74fc404fbea admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:46.331672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4ecc6-f3da-472c-9b9e-e74fc404fbea admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:46.331945 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4ecc6-f3da-472c-9b9e-e74fc404fbea admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:46.332594 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4ecc6-f3da-472c-9b9e-e74fc404fbea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:46.334750 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4ecc6-f3da-472c-9b9e-e74fc404fbea admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:46.344719 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1471/2926] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:46.348632 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586172a-04e9-4466-ba5d-3285f3bb9ea9 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:46.358949 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586172a-04e9-4466-ba5d-3285f3bb9ea9 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:46.363785 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacec01c-f059-4e45-834c-9c3eb4995f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:46.364380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacec01c-f059-4e45-834c-9c3eb4995f16 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:46.364658 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacec01c-f059-4e45-834c-9c3eb4995f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:46.364932 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacec01c-f059-4e45-834c-9c3eb4995f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:46.365657 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacec01c-f059-4e45-834c-9c3eb4995f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:46.367273 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacec01c-f059-4e45-834c-9c3eb4995f16 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:46.378610 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1456/2927] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:04:46.382715 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4ecc6-f3da-472c-9b9e-e74fc404fbea admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:46.392508 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4ecc6-f3da-472c-9b9e-e74fc404fbea admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:46.397670 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7757cb-ca66-464b-940b-5525446eccbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:46.398333 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7757cb-ca66-464b-940b-5525446eccbd admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:46.398613 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7757cb-ca66-464b-940b-5525446eccbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:46.398957 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7757cb-ca66-464b-940b-5525446eccbd admin admin] PATH_INFO: `/v3/projects/2e1fb35e7be541c8bec40499ed3ac2d2/users/e76f24263ef74c28b99e1f882071bf5b/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:46.413728 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7757cb-ca66-464b-940b-5525446eccbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1fb35e7be541c8bec40499ed3ac2d2, user_id=e76f24263ef74c28b99e1f882071bf5b, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:46.416369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7757cb-ca66-464b-940b-5525446eccbd admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:46.429814 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1472/2928] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:04:46 2022] PUT /identity/v3/projects/2e1fb35e7be541c8bec40499ed3ac2d2/users/e76f24263ef74c28b99e1f882071bf5b/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 09:04:46.434728 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adff5a-bf26-468e-927e-7ae6e935a062 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:46.435230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adff5a-bf26-468e-927e-7ae6e935a062 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:46.435230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adff5a-bf26-468e-927e-7ae6e935a062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:46.453563 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-40adff5a-bf26-468e-927e-7ae6e935a062 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:46.454815 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-40adff5a-bf26-468e-927e-7ae6e935a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:46.512324 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1457/2929] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:46.522440 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7757cb-ca66-464b-940b-5525446eccbd admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:46.524091 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-bc7757cb-ca66-464b-940b-5525446eccbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:46.580348 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7757cb-ca66-464b-940b-5525446eccbd admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:46.584653 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f8ea0-6d0a-4861-ba49-b89085983dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:46.585253 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f8ea0-6d0a-4861-ba49-b89085983dcd service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:46.585531 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f8ea0-6d0a-4861-ba49-b89085983dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:46.585835 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f8ea0-6d0a-4861-ba49-b89085983dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:46.594872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f8ea0-6d0a-4861-ba49-b89085983dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:46.596315 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f8ea0-6d0a-4861-ba49-b89085983dcd service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:46.605536 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1473/2930] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:47.210869 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16d006-8fd8-4718-9e8d-91338ed56277 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:47.211541 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16d006-8fd8-4718-9e8d-91338ed56277 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:47.211831 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16d006-8fd8-4718-9e8d-91338ed56277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:47.239784 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-2d16d006-8fd8-4718-9e8d-91338ed56277 None None] MFA Rules not processed for user `e76f24263ef74c28b99e1f882071bf5b`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:47.241435 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-2d16d006-8fd8-4718-9e8d-91338ed56277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:47.305251 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1458/2931] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:47.316072 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f8ea0-6d0a-4861-ba49-b89085983dcd service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:04:47.317497 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-d77f8ea0-6d0a-4861-ba49-b89085983dcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:47.379590 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:47.401602 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f8ea0-6d0a-4861-ba49-b89085983dcd service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:04:47.405335 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d8b59-e517-4b56-b938-b6df08042d20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:04:47.406021 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d8b59-e517-4b56-b938-b6df08042d20 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:47.406302 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d8b59-e517-4b56-b938-b6df08042d20 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:47.406578 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d8b59-e517-4b56-b938-b6df08042d20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:47.415493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d8b59-e517-4b56-b938-b6df08042d20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:04:47.416928 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d8b59-e517-4b56-b938-b6df08042d20 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:04:47.426042 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1474/2932] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:04:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:04:48.959540 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8289139-3d76-4f2c-a95f-e7080ef5ca3f None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:04:48.960073 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8289139-3d76-4f2c-a95f-e7080ef5ca3f None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:04:48.960193 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8289139-3d76-4f2c-a95f-e7080ef5ca3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:04:48.979392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-d8289139-3d76-4f2c-a95f-e7080ef5ca3f None None] MFA Rules not processed for user `e76f24263ef74c28b99e1f882071bf5b`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:04:48.980566 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d8289139-3d76-4f2c-a95f-e7080ef5ca3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:04:48.987250 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1459/2933] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:04:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:04:57.421672 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=11 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:04:58.973526 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:05:01.604180 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d8b59-e517-4b56-b938-b6df08042d20 service cinder] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:05:01.605654 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a43d8b59-e517-4b56-b938-b6df08042d20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:05:01.723792 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d8b59-e517-4b56-b938-b6df08042d20 service cinder] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:05:01.729321 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cf70a-9e00-4e22-b246-2ae2e3293d0f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:05:01.729942 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cf70a-9e00-4e22-b246-2ae2e3293d0f service placement] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:05:01.730226 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cf70a-9e00-4e22-b246-2ae2e3293d0f service placement] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:05:01.730501 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cf70a-9e00-4e22-b246-2ae2e3293d0f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:05:01.732235 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-e44cf70a-9e00-4e22-b246-2ae2e3293d0f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:05:01.747123 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cf70a-9e00-4e22-b246-2ae2e3293d0f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:05:01.748726 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cf70a-9e00-4e22-b246-2ae2e3293d0f service placement] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:05:01.760327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1475/2934] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:05:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:05:11.754381 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=41 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:05:47.754925 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8289139-3d76-4f2c-a95f-e7080ef5ca3f None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:05:47.764390 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8289139-3d76-4f2c-a95f-e7080ef5ca3f None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:05:47.767870 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270fee7-efe9-4a63-b1e4-ca68bbb905b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:05:47.768456 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270fee7-efe9-4a63-b1e4-ca68bbb905b6 service cinder] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:05:47.768733 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270fee7-efe9-4a63-b1e4-ca68bbb905b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:05:47.769003 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270fee7-efe9-4a63-b1e4-ca68bbb905b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:05:47.770731 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-8270fee7-efe9-4a63-b1e4-ca68bbb905b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:05:47.862391 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270fee7-efe9-4a63-b1e4-ca68bbb905b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:05:47.863856 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270fee7-efe9-4a63-b1e4-ca68bbb905b6 service cinder] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:05:47.883359 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1460/2935] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:05:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:05:50.473397 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cf70a-9e00-4e22-b246-2ae2e3293d0f service placement] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:05:50.483492 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cf70a-9e00-4e22-b246-2ae2e3293d0f service placement] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:05:50.487074 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de74b1-793e-4aa0-92c1-496c27cb9bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:05:50.487660 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de74b1-793e-4aa0-92c1-496c27cb9bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:05:50.487938 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de74b1-793e-4aa0-92c1-496c27cb9bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:05:50.488206 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de74b1-793e-4aa0-92c1-496c27cb9bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:05:50.489921 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-a8de74b1-793e-4aa0-92c1-496c27cb9bd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:05:50.503420 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de74b1-793e-4aa0-92c1-496c27cb9bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:05:50.504841 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de74b1-793e-4aa0-92c1-496c27cb9bd4 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:05:50.515545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1476/2936] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:05:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:05:57.880791 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=46 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:06:00.510684 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=6 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:06:02.579146 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be73c09-eeab-4c68-8488-1e98e6d01c9a None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:06:02.579695 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be73c09-eeab-4c68-8488-1e98e6d01c9a None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:06:02.579813 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be73c09-eeab-4c68-8488-1e98e6d01c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:06:02.598833 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-1be73c09-eeab-4c68-8488-1e98e6d01c9a None None] MFA Rules not processed for user `e76f24263ef74c28b99e1f882071bf5b`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:06:02.600030 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1be73c09-eeab-4c68-8488-1e98e6d01c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:06:02.606840 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1461/2937] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:06:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:06:02.839177 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de74b1-793e-4aa0-92c1-496c27cb9bd4 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:06:02.847857 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de74b1-793e-4aa0-92c1-496c27cb9bd4 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:06:02.851446 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f0099-9d8e-4b1a-9f3a-90642ae5a782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:06:02.852038 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f0099-9d8e-4b1a-9f3a-90642ae5a782 service neutron] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:06:02.852318 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f0099-9d8e-4b1a-9f3a-90642ae5a782 service neutron] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:06:02.852592 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f0099-9d8e-4b1a-9f3a-90642ae5a782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:06:02.854369 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-ba2f0099-9d8e-4b1a-9f3a-90642ae5a782 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:06:02.864160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f0099-9d8e-4b1a-9f3a-90642ae5a782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:06:02.865585 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f0099-9d8e-4b1a-9f3a-90642ae5a782 service neutron] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:06:02.874078 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1477/2938] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:06:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:06:12.593432 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=3 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:06:12.869910 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=6 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:07:06.258660 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be73c09-eeab-4c68-8488-1e98e6d01c9a None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:06.269717 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be73c09-eeab-4c68-8488-1e98e6d01c9a None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:06.275230 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de702ad9-dc12-4ae0-aef5-462624f097df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:06.275855 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de702ad9-dc12-4ae0-aef5-462624f097df admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:06.276195 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de702ad9-dc12-4ae0-aef5-462624f097df admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:06.276395 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de702ad9-dc12-4ae0-aef5-462624f097df admin admin] PATH_INFO: `/v3/users/e76f24263ef74c28b99e1f882071bf5b` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:06.278097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de702ad9-dc12-4ae0-aef5-462624f097df admin admin] RBAC: Authorizing `identity:delete_user(user_id=e76f24263ef74c28b99e1f882071bf5b)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:06.280088 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de702ad9-dc12-4ae0-aef5-462624f097df admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:06.325392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-de702ad9-dc12-4ae0-aef5-462624f097df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e76f24263ef74c28b99e1f882071bf5b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:06.326084 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-de702ad9-dc12-4ae0-aef5-462624f097df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e76f24263ef74c28b99e1f882071bf5b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:06.337798 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-de702ad9-dc12-4ae0-aef5-462624f097df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e76f24263ef74c28b99e1f882071bf5b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:06.345340 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-de702ad9-dc12-4ae0-aef5-462624f097df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e76f24263ef74c28b99e1f882071bf5b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:06.346007 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-de702ad9-dc12-4ae0-aef5-462624f097df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e76f24263ef74c28b99e1f882071bf5b'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:06.359163 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1462/2939] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 09:07:06 2022] DELETE /identity/v3/users/e76f24263ef74c28b99e1f882071bf5b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:07:06.540777 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f0099-9d8e-4b1a-9f3a-90642ae5a782 service neutron] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:06.552936 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f0099-9d8e-4b1a-9f3a-90642ae5a782 service neutron] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:06.608712 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:06.609339 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:06.609619 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:06.609918 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179 admin admin] PATH_INFO: `/v3/projects/2e1fb35e7be541c8bec40499ed3ac2d2` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:06.611025 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e1fb35e7be541c8bec40499ed3ac2d2)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:06.612925 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:06.658179 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e1fb35e7be541c8bec40499ed3ac2d2', 'request_id': 'req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:06.664468 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e1fb35e7be541c8bec40499ed3ac2d2', 'request_id': 'req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:06.670145 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e1fb35e7be541c8bec40499ed3ac2d2', 'request_id': 'req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:06.682464 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179 admin admin] The token cache is being invalidate because project 2e1fb35e7be541c8bec40499ed3ac2d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:07:06.682756 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:06.683375 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-5b8fc9a3-0aa3-49d4-8945-5ce3c65f6179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:06.684847 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1478/2940] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 09:07:06 2022] DELETE /identity/v3/projects/2e1fb35e7be541c8bec40499ed3ac2d2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:07:06.689749 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a4297-842f-454b-bd73-16c5a5917d3e None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:06.690037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a4297-842f-454b-bd73-16c5a5917d3e None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:06.690313 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a4297-842f-454b-bd73-16c5a5917d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:06.710314 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-468a4297-842f-454b-bd73-16c5a5917d3e None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:07:06.711777 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-468a4297-842f-454b-bd73-16c5a5917d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:07:06.772684 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1463/2941] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:07:06.781069 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5cf2f-0f27-4292-a225-6f597664cef5 None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:06.781364 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5cf2f-0f27-4292-a225-6f597664cef5 None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:06.781647 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5cf2f-0f27-4292-a225-6f597664cef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:06.801049 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-f6c5cf2f-0f27-4292-a225-6f597664cef5 None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:07:06.802291 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-f6c5cf2f-0f27-4292-a225-6f597664cef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:07:06.810160 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1479/2942] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:07:06.814189 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a4297-842f-454b-bd73-16c5a5917d3e None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:06.824501 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a4297-842f-454b-bd73-16c5a5917d3e None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:06.829199 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c74fd-7423-4382-94fa-b4a770d6f5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:06.829832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c74fd-7423-4382-94fa-b4a770d6f5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:06.830115 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c74fd-7423-4382-94fa-b4a770d6f5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:06.830392 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c74fd-7423-4382-94fa-b4a770d6f5b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:06.831037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c74fd-7423-4382-94fa-b4a770d6f5b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87946) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 09:07:06.831343 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c74fd-7423-4382-94fa-b4a770d6f5b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:06.832940 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c74fd-7423-4382-94fa-b4a770d6f5b7 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:06.845271 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1464/2943] 10.222.0.26 () {58 vars in 1153 bytes} [Tue Oct 4 09:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 09:07:06.849320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5cf2f-0f27-4292-a225-6f597664cef5 None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:06.858558 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5cf2f-0f27-4292-a225-6f597664cef5 None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:06.863220 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ad6e2-ac90-41a5-8ddd-ecce397a0186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:06.863807 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ad6e2-ac90-41a5-8ddd-ecce397a0186 admin admin] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:06.864118 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ad6e2-ac90-41a5-8ddd-ecce397a0186 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:06.864394 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ad6e2-ac90-41a5-8ddd-ecce397a0186 admin admin] PATH_INFO: `/v3/projects` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:06.865029 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ad6e2-ac90-41a5-8ddd-ecce397a0186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:06.866980 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ad6e2-ac90-41a5-8ddd-ecce397a0186 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:06.903978 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1480/2944] 10.222.0.26 () {60 vars in 1153 bytes} [Tue Oct 4 09:07:06 2022] POST /identity/v3/projects => generated 397 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 09:07:06.907993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c74fd-7423-4382-94fa-b4a770d6f5b7 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:06.917875 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c74fd-7423-4382-94fa-b4a770d6f5b7 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:06.973481 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f3358-2b04-4133-8015-cc3cc6f23094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:06.974180 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f3358-2b04-4133-8015-cc3cc6f23094 admin admin] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:06.974466 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f3358-2b04-4133-8015-cc3cc6f23094 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:06.974743 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f3358-2b04-4133-8015-cc3cc6f23094 admin admin] PATH_INFO: `/v3/users` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:06.975375 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f3358-2b04-4133-8015-cc3cc6f23094 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:06.977274 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f3358-2b04-4133-8015-cc3cc6f23094 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:06.985165 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:07:06.999613 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1465/2945] 10.222.0.26 () {60 vars in 1144 bytes} [Tue Oct 4 09:07:06 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 09:07:07.003747 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ad6e2-ac90-41a5-8ddd-ecce397a0186 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:07.014545 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ad6e2-ac90-41a5-8ddd-ecce397a0186 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:07.019178 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4b2de-2f86-46f0-9bc6-ce4f5412600d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:07.019769 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4b2de-2f86-46f0-9bc6-ce4f5412600d admin admin] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:07.020052 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4b2de-2f86-46f0-9bc6-ce4f5412600d admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:07.020328 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4b2de-2f86-46f0-9bc6-ce4f5412600d admin admin] PATH_INFO: `/v3/roles` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:07.020968 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4b2de-2f86-46f0-9bc6-ce4f5412600d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:07.022611 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4b2de-2f86-46f0-9bc6-ce4f5412600d admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:07.032462 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1481/2946] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:07:07.036380 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f3358-2b04-4133-8015-cc3cc6f23094 admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:07.046062 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f3358-2b04-4133-8015-cc3cc6f23094 admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:07.050819 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854433a3-3c1e-4a56-b05e-db438d157619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:07.051418 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854433a3-3c1e-4a56-b05e-db438d157619 admin admin] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:07.051697 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854433a3-3c1e-4a56-b05e-db438d157619 admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:07.051966 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854433a3-3c1e-4a56-b05e-db438d157619 admin admin] PATH_INFO: `/v3/roles` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:07.052604 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854433a3-3c1e-4a56-b05e-db438d157619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:07.054341 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854433a3-3c1e-4a56-b05e-db438d157619 admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:07.066037 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1466/2947] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Oct 4 09:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 09:07:07.070541 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4b2de-2f86-46f0-9bc6-ce4f5412600d admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:07.081327 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4b2de-2f86-46f0-9bc6-ce4f5412600d admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:07.086281 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec2a20-a4ea-4465-bf8c-e2d83834a1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:07.086952 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2a20-a4ea-4465-bf8c-e2d83834a1c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:07.087237 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2a20-a4ea-4465-bf8c-e2d83834a1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:07.087517 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2a20-a4ea-4465-bf8c-e2d83834a1c2 admin admin] PATH_INFO: `/v3/projects/c5aff5a5050c46329504ac3b8cf84625/users/b545343e6d694541b32244f4e5e45259/roles/efddf5f5222644b18b6836117621adf8` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:07.102390 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec2a20-a4ea-4465-bf8c-e2d83834a1c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5aff5a5050c46329504ac3b8cf84625, user_id=b545343e6d694541b32244f4e5e45259, role_id=efddf5f5222644b18b6836117621adf8)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:07.105077 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec2a20-a4ea-4465-bf8c-e2d83834a1c2 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:07.117674 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1482/2948] 10.222.0.26 () {60 vars in 1483 bytes} [Tue Oct 4 09:07:07 2022] PUT /identity/v3/projects/c5aff5a5050c46329504ac3b8cf84625/users/b545343e6d694541b32244f4e5e45259/roles/efddf5f5222644b18b6836117621adf8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:07:07.128356 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48707099-daba-407b-ba87-70ef4b692aa2 None None] REQUEST_METHOD: `POST` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:07.128659 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48707099-daba-407b-ba87-70ef4b692aa2 None None] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:07.128930 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48707099-daba-407b-ba87-70ef4b692aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:07.156299 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.auth.core [None req-48707099-daba-407b-ba87-70ef4b692aa2 None None] MFA Rules not processed for user `b545343e6d694541b32244f4e5e45259`. Rule list: `[]` (Enabled: `True`). {{(pid=87946) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:07:07.157524 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-48707099-daba-407b-ba87-70ef4b692aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:07:07.222238 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1467/2949] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:07:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:07:07.234700 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec2a20-a4ea-4465-bf8c-e2d83834a1c2 admin admin] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:07.236704 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-0fec2a20-a4ea-4465-bf8c-e2d83834a1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:07:07.257103 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:07:07.295421 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec2a20-a4ea-4465-bf8c-e2d83834a1c2 admin admin] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:07.299048 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b10a4-f624-4cbb-89f0-e7a03c2ec3c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f224ed840244faea117fe97c29c26d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:07.299654 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b10a4-f624-4cbb-89f0-e7a03c2ec3c7 service cinder] REQUEST_METHOD: `GET` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:07.300071 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b10a4-f624-4cbb-89f0-e7a03c2ec3c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:07.300354 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b10a4-f624-4cbb-89f0-e7a03c2ec3c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:07.309130 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b10a4-f624-4cbb-89f0-e7a03c2ec3c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:07.310599 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b10a4-f624-4cbb-89f0-e7a03c2ec3c7 service cinder] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:07.319309 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1483/2950] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:07:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:07:15.379827 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48707099-daba-407b-ba87-70ef4b692aa2 None None] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:15.442208 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48707099-daba-407b-ba87-70ef4b692aa2 None None] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:15.446853 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:15.447480 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:15.447765 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:15.448041 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] PATH_INFO: `/v3/users/b545343e6d694541b32244f4e5e45259` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:15.449768 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] RBAC: Authorizing `identity:delete_user(user_id=b545343e6d694541b32244f4e5e45259)` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:15.451690 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:15.497547 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b545343e6d694541b32244f4e5e45259'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:15.498236 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b545343e6d694541b32244f4e5e45259'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:15.509903 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b545343e6d694541b32244f4e5e45259'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:15.514828 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:07:15.518097 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b545343e6d694541b32244f4e5e45259'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:15.519022 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.notifications [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b545343e6d694541b32244f4e5e45259'} {{(pid=87946) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:15.531975 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1468/2951] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Oct 4 09:07:15 2022] DELETE /identity/v3/users/b545343e6d694541b32244f4e5e45259 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:07:15.536444 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd8fa-dfaa-4dbd-8c81-2cec9698031a None None] REQUEST_METHOD: `POST` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:15.536909 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd8fa-dfaa-4dbd-8c81-2cec9698031a None None] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:15.537034 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd8fa-dfaa-4dbd-8c81-2cec9698031a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:15.555798 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.auth.core [None req-98fdd8fa-dfaa-4dbd-8c81-2cec9698031a None None] MFA Rules not processed for user `f84b5144054d4e158a838b4ae551996f`. Rule list: `[]` (Enabled: `True`). {{(pid=87945) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 09:07:15.557020 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-98fdd8fa-dfaa-4dbd-8c81-2cec9698031a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:07:15.615605 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1484/2952] 10.222.0.26 () {58 vars in 958 bytes} [Tue Oct 4 09:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 09:07:15.625918 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:15.627339 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:07:15.683696 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364edcb1-7bd9-4a04-b20a-f8539b24d6af admin admin] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:15.687328 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c425319-b560-4b89-a72a-54dc0bda6098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50d3b75cbc5c4d1686f4d93248e04746', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:15.687929 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c425319-b560-4b89-a72a-54dc0bda6098 service neutron] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:15.688210 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c425319-b560-4b89-a72a-54dc0bda6098 service neutron] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:15.688482 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c425319-b560-4b89-a72a-54dc0bda6098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:15.697346 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c425319-b560-4b89-a72a-54dc0bda6098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:15.698832 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c425319-b560-4b89-a72a-54dc0bda6098 service neutron] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:15.708060 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1469/2953] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:07:15.974044 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdd8fa-dfaa-4dbd-8c81-2cec9698031a None None] Authenticating user token {{(pid=87945) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:07:15.975493 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.fernet_utils [None req-98fdd8fa-dfaa-4dbd-8c81-2cec9698031a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87945) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:07:15.987871 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdd8fa-dfaa-4dbd-8c81-2cec9698031a None None] Validating token access rules against request {{(pid=87945) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:07:15.992880 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21403c33-9026-43ae-81fa-e0c99413bd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84b5144054d4e158a838b4ae551996f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47b3fa8678ad4f779e78d8c72364fac2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87945) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:07:15.993514 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21403c33-9026-43ae-81fa-e0c99413bd60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:07:15.994046 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21403c33-9026-43ae-81fa-e0c99413bd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:07:15.994350 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21403c33-9026-43ae-81fa-e0c99413bd60 admin admin] PATH_INFO: `/v3/projects/c5aff5a5050c46329504ac3b8cf84625` {{(pid=87945) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:07:15.996084 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21403c33-9026-43ae-81fa-e0c99413bd60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5aff5a5050c46329504ac3b8cf84625)` {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:07:15.998320 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21403c33-9026-43ae-81fa-e0c99413bd60 admin admin] RBAC: Authorization granted {{(pid=87945) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:07:16.046428 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-21403c33-9026-43ae-81fa-e0c99413bd60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5aff5a5050c46329504ac3b8cf84625', 'request_id': 'req-21403c33-9026-43ae-81fa-e0c99413bd60'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:16.052476 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-21403c33-9026-43ae-81fa-e0c99413bd60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5aff5a5050c46329504ac3b8cf84625', 'request_id': 'req-21403c33-9026-43ae-81fa-e0c99413bd60'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:16.057872 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-21403c33-9026-43ae-81fa-e0c99413bd60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5aff5a5050c46329504ac3b8cf84625', 'request_id': 'req-21403c33-9026-43ae-81fa-e0c99413bd60'} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:16.070006 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-21403c33-9026-43ae-81fa-e0c99413bd60 admin admin] The token cache is being invalidate because project c5aff5a5050c46329504ac3b8cf84625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87945) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 09:07:16.070294 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-21403c33-9026-43ae-81fa-e0c99413bd60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:16.070913 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG keystone.notifications [None req-21403c33-9026-43ae-81fa-e0c99413bd60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87945) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 09:07:16.072380 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: [pid: 87945|app: 0|req: 1485/2954] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Oct 4 09:07:15 2022] DELETE /identity/v3/projects/c5aff5a5050c46329504ac3b8cf84625 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 09:07:25.703954 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=30,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:07:26.067029 dv-cinder-860212-1-iscsi devstack@keystone.service[87945]: DEBUG dbcounter [-] [87945] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87945) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 09:10:05.606379 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c425319-b560-4b89-a72a-54dc0bda6098 service neutron] Authenticating user token {{(pid=87946) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 09:10:05.607770 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-1c425319-b560-4b89-a72a-54dc0bda6098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:05.676159 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c425319-b560-4b89-a72a-54dc0bda6098 service neutron] Validating token access rules against request {{(pid=87946) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 09:10:05.734105 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3f0ba-6931-43ef-8869-0eee7ba66e37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abd158648914409f87ce0599843f0fcf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa3df14a62714f6bab2acdfc979b655b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87946) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 09:10:05.734710 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3f0ba-6931-43ef-8869-0eee7ba66e37 service placement] REQUEST_METHOD: `GET` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 09:10:05.734993 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3f0ba-6931-43ef-8869-0eee7ba66e37 service placement] SCRIPT_NAME: `/identity` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 09:10:05.735269 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3f0ba-6931-43ef-8869-0eee7ba66e37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87946) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 09:10:05.737004 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.fernet_utils [None req-d3d3f0ba-6931-43ef-8869-0eee7ba66e37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87946) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 09:10:05.748076 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3f0ba-6931-43ef-8869-0eee7ba66e37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 09:10:05.749646 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3f0ba-6931-43ef-8869-0eee7ba66e37 service placement] RBAC: Authorization granted {{(pid=87946) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 09:10:05.759953 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: [pid: 87946|app: 0|req: 1470/2955] 10.222.0.26 () {60 vars in 1365 bytes} [Tue Oct 4 09:10:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 09:10:15.755175 dv-cinder-860212-1-iscsi devstack@keystone.service[87946]: DEBUG dbcounter [-] [87946] Writing DB stats keystone:SELECT=58 {{(pid=87946) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}